[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: it looks like bug:(
From: |
Micah Cowan |
Subject: |
Re: it looks like bug:( |
Date: |
Thu, 18 Oct 2007 14:06:12 -0700 |
User-agent: |
Thunderbird 1.5.0.13 (X11/20070824) |
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Andrew J. Schorr wrote:
> On Thu, Oct 18, 2007 at 09:28:27AM -0700, Micah Cowan wrote:
>> Try changing ‘print w’ to ‘printf("%.24g\n", w)’, or just set OFMT to a
>> value of "%.24g". OFMT is the conversion specification used to convert
>> numbers to strings for output; CONVFMT is used to convert numbers to
>> strings when required by an expression.
>
> Actually, the IEEE 754 double-precision floating-point format that gawk
> typically uses has about 16 digits of precision, so printing "%.24g" goes well
> beyond the actual precision of the floating-point representation.
Right. I was being generous. :)
> The easiest way to see the problem is simply to print w-100
> instead of printing w:
>
> bash-3.1$ gawk 'BEGIN{
> > w = 0.0
> > w = w + 0.15
> > w = w + 99.45
> > w = w + 0.4
> > print w-100
> > }'
> 1.42109e-14
Illuminates the issue quite clearly indeed.
- --
Micah J. Cowan
Programmer, musician, typesetting enthusiast, gamer...
http://micah.cowan.name/
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFHF8rE7M8hyUobTrERCJ1VAJ9GXKh6igsjO4+1Rknp7RMmYQmekACgj7We
MVIimNl++InzJTc1avA87LI=
=Jrhu
-----END PGP SIGNATURE-----