[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: it looks like bug:(
From: |
Andrew J. Schorr |
Subject: |
Re: it looks like bug:( |
Date: |
Thu, 18 Oct 2007 17:03:03 -0400 |
User-agent: |
Mutt/1.4.2.2i |
On Thu, Oct 18, 2007 at 09:28:27AM -0700, Micah Cowan wrote:
> Try changing ‘print w’ to ‘printf("%.24g\n", w)’, or just set OFMT to a
> value of "%.24g". OFMT is the conversion specification used to convert
> numbers to strings for output; CONVFMT is used to convert numbers to
> strings when required by an expression.
Actually, the IEEE 754 double-precision floating-point format that gawk
typically uses has about 16 digits of precision, so printing "%.24g" goes well
beyond the actual precision of the floating-point representation.
The easiest way to see the problem is simply to print w-100
instead of printing w:
bash-3.1$ gawk 'BEGIN{
> w = 0.0
> w = w + 0.15
> w = w + 99.45
> w = w + 0.4
> print w-100
> }'
1.42109e-14
You can find lots more info on IEEE 754 arithmetic by searching the internet.
Here, for example, is one document that explains the situation:
http://docs.sun.com/source/806-3568/ncg_math.html
Regards,
Andy