[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Help-smalltalk] Re: BUG gst2.2: Float asExactFraction asFloat inexact

From: nicolas cellier
Subject: [Help-smalltalk] Re: BUG gst2.2: Float asExactFraction asFloat inexact
Date: Sat, 27 Jan 2007 03:59:08 +0100
User-agent: Thunderbird (X11/20060516)

Post Scriptum:

This post does only make sense if you apply first the patch i proposed with subject:
   BUG gst2.2: Float asExactFraction broken

I published above first but gmane seems to scramble my posts...

nicolas cellier a écrit :

Converting a Fraction to a Float is generally inexact. That's OK. Float are inexact (rounded).

But that should not be the case for Fraction with 53bit numerator and denominator power of two, as obtained with asExactFraction.

Unfortunately, current implementation of both Largeinteger>>asFloat and Fraction>>asFloat will cause successive round off errors...

| x |
x := 0.510000001d-20.
^x asExactFraction = x

More over, denormalized number (gradual underflow) will convert back to zero...

| x |
x := 1.1d-310.
^x asExactFraction asFloat = 0

Nicolas Cellier

reply via email to

[Prev in Thread] Current Thread [Next in Thread]