Re: Bug report: sort.c or AIX compiler

[Jim Meyering]

Jim Meyering <address@hidden> wrote:
> "Lemley James - jlemle" <address@hidden> wrote:
>> when compiling coreutils 5.90 on AIX 5.3 in 64 bit mode
>> (OBJECT_MODE=64),
>>
>> sort dumps core at line 1724 or 1725 (coreutils 5.90).
>>
>> It works fine in 32-bit mode, and debugging statements I add make it
>> seem like the code is doing the right thing.
>>
>> Same thing is present in coreutils 5.30, and probably has been in sort.c
>> since it was written, but that's a wild guess.
>>
>> When this change is made, it runs just fine (well, it works for at least
>> one input) after being compiled in 64-bit mode:
>>
>> 1723c1723
>>
>> <       bool swap = (0 < compare (&lines[-1], &lines[-2]));
>>
>> ---
>>
>>>       int swap = (0 < compare (&lines[-1], &lines[-2]));

Thanks again.
I've done as you suggest:

2005-10-05  Jim Meyering  <address@hidden>

        * src/sort.c (sortlines_temp): Declare temporary as `int' rather
        than `bool' to work around AIX 5.3 compiler bug in 64-bit mode.
        From James Lemley.

Index: src/sort.c
===================================================================
RCS file: /fetish/cu/src/sort.c,v
retrieving revision 1.324
retrieving revision 1.326
diff -u -p -r1.324 -r1.326
--- src/sort.c  1 Sep 2005 19:31:10 -0000       1.324
+++ src/sort.c  5 Oct 2005 18:56:29 -0000       1.326
@@ -1720,7 +1720,9 @@ sortlines_temp (struct line *lines, size
 {
   if (nlines == 2)
     {
-      bool swap = (0 < compare (&lines[-1], &lines[-2]));
+      /* Declare `swap' as int, not bool, to work around a bug in
+         an AIX 5.3 compiler in 64-bit mode.  */
+      int swap = (0 < compare (&lines[-1], &lines[-2]));
       temp[-1] = lines[-1 - swap];
       temp[-2] = lines[-2 + swap];
     }