RE: using make to run binaries

[Dave Korn]

On 19 September 2007 14:43, Andreas Maunz wrote:

> Hi Dave,
> 
>>   Here's a thought: does it still happen if you compile your program with
>> -O0, i.e. no optimisation?
> 
> this actually did the trick! :-) Can you elaborate a little on that? How
> specifically does this affect the execution of the program? We tried
> 'rake' in the meantime and it produced the manual result (without -O0). So
> it looks like some make-specific oddity.


  No, I'm afraid not.  It means it's almost certain now to be a bug in your
program that introduces undefined behaviour.  I don't have time to write a
detailed thesis right now (I'm at work in the middle of the afternoon), but
the things that are undefined are generally highly sensitive to optimisation;
here's just a quick example:

  Consider using a variable before it's been set.  With optimisation off, that
variable will be stored in a stack slot in the stack frame for the function.
If the stack memory space is zeroed ahead of time, then this code might be
entirely reliable, behaving just as designed, owing to the stack space having
been fortuitously zeroed and this implicitly initialising the variable to
zero.  Hooray, the program works, but only by good fortune.  Now imagine what
happens when you add -O2 to your optimisation flags.  This time, the compiler
is going out of its way to avoid using memory, since memory accesses are slow.
So it puts all uses of the variable during the subroutine into one of the
CPU's registers.  Now, unlike memory space, registers don't get zeroed ahead
of time - there's no mallocing to get yourself a register - so this time, when
you run the same program compiled with all the same environment variables and
command-line options, it comes to the same subroutine.  But this time, the
uninitialised variable isn't zero, it's some long hex value, whatever value
just happened to be left in that CPU register previously by the last
instructions to use it.  Bang!  Same program, same commandline, same
environment - but different behaviour.

  The rules of undefined behaviour are generally specified in a way that give
the compiler latitude to make optimisations.  An example: shifting a value by
more than the number of bits in the wordsize is undefined.  That means that an
optimising compiler can represent X >> Y by a single CPU shift instructions.
Different cpus will behave differently if you load X into one register and Y
into another and tell it to shift X by Y amount.  In the 32-bit case, some of
them will truncate Y at 31.  Some of them will AND Y with 31.  Some of them
will return a result of zero.  The compiler's ability to optimise would be
crippled if it had to insert checks and tests and branches to cover all these
possiblities, as compared to just inserting a shift instruction and telling
the coder that there are restrictions on the use of the >> operator and it'll
go wrong if you shift by 32 or more.  So that's the trade-off in the standard:
undefined behaviour means more chances for optimisation in valid programs.
And the consequnce in real life is that undefined behaviour most often crops
up when you turn on optimisation, and often doesn't trigger (although the
program is still incorrect) without it.

  You need to start looking at your program.  If you're using advanced fp
maths (which I suspect you are from your description) and it's on an x86 (or
compatible) cpu, you could very well have run into the legendary
8087-fpu-excess-precision bug aka PR323:

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=323

  This one often appears with optimisation, when floating point intermediate
results are cached in extended-precision registers, and often disappears at
-O0 (or with `-ffloat-store'), when all intermediate results are stored in
temporaries on the stack, forcing the excess precision to be truncated away.

  I'm as certain as it's possible to be (without having a copy of your code
and debugging it for you!) that the problem is in your program, and that it's
just a coincidence that it happens to show when launched from make and not
from the commandline.


    cheers,
      DaveK
-- 
Can't think of a witty .sigline today....