Re: [Help-bash] Uniq identical lines not matching

[Matthew Cengia]

On 2016-02-17 23:48, Christopher Maier wrote:
> Hi,
> 
> I've run into an odd problem with uniq, which I've simplified to the below 
> example. When trying to remove duplicates from identical consecutive text 
> lines where there is a special byte like \xFF, it does not consider most of 
> the lines to be identical and passes them through. What could be causing this?
> 
> address@hidden:~> echo $BASH_VERSION
> 3.2.51(1)-release
> 
> address@hidden:~> echo 'aabaaa' | sed 's|b|\xFF|g' | awk '{print $1; print 
> $1; print $1; print $1; print $1;}' | uniq -c
>       2 aaÿaaa
>       1 aaÿaaa
>       1 aaÿaaa
>       1 aaÿaaa

Quite sure this is a problem within uniq, not bash:

| address@hidden:tmp$ echo 'aabaaa' | sed 's|b|\xFF|g' | awk '{print $1; print 
$1; print $1; print $1; print $1;}' | uniq -c
|       5 aa�aaa
| address@hidden:tmp$ uniq --version
| uniq (GNU coreutils) 8.13
| Copyright (C) 2011 Free Software Foundation, Inc.
| License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
| This is free software: you are free to change and redistribute it.
| There is NO WARRANTY, to the extent permitted by law.
| 
| Written by Richard M. Stallman and David MacKenzie.

-- 
Regards,
Matthew Cengia

signature.asc
Description: Digital signature