help-smalltalk
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Help-smalltalk] [bug] LargePositiveInteger asFloat: does not round corr


From: Nicolas Cellier
Subject: [Help-smalltalk] [bug] LargePositiveInteger asFloat: does not round correctly
Date: Sat, 06 Jun 2009 15:11:05 -0700

Issue status update for http://smalltalk.gnu.org/node/309 Post a follow up: http://smalltalk.gnu.org/project/comments/add/309

Project:      GNU Smalltalk
Version:      <none>
Component:    Base classes
Category:     bug reports
Priority:     normal
Assigned to:  Unassigned
Reported by:  nice
Updated by:   nice
Status:       active

Paolo, the simplification you made are great!
But you just swapped the lowBit test...

Example:

self assert: ((1 bitShift: 53+100) + (1 bitShift: 100) + 1) asFloatD
truncated lowBit = 102.

Number of truncated bits is 101.
Carry is 1.
Truncated mantissa candidate is even (1 bitShift: 52).
But there are other trailing bits behind the carry (self lowBit <
nTruncatedBits), so the truncated part > 0.5 ulp, so the mantissa must
be rounded upper.

Please correct.
Cheers






reply via email to

[Prev in Thread] Current Thread [Next in Thread]