[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Output precision
|
From: |
Martin Helm |
|
Subject: |
Re: Output precision |
|
Date: |
Sun, 15 Apr 2012 13:02:07 +0200 |
|
User-agent: |
Mozilla/5.0 (X11; Linux x86_64; rv:11.0) Gecko/20120328 Thunderbird/11.0.1 |
Am 15.04.2012 12:40, schrieb JuanPi:
> x = 987654321.123456789;
x is a double precision IEEE 754 floating point variable, you can show
the precision by using eps.
octave:1> x = 987654321.123456789;
octave:2> x*eps
ans = 2.1930e-07
shows that you have to expect that at least the 7th digit after the
decimal point is wrong and this is exactly what you see.
For your other example you run into the typical situation that some
values which can be represented exactly in decimal representation can
not be represented exactly in binary representation.