[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: GAWK adding big numbers wrong
From: |
laura fairhead |
Subject: |
Re: GAWK adding big numbers wrong |
Date: |
Mon, 13 May 2002 20:27:59 GMT |
[]
>>
>>Package Version
>>bash 2.05a-3
>>gawk 3.0.4-1
>>
>>System is:
>>Microsoft Windows 98 4.10.2222 A
>>GenuineIntel x86 Family 6 Model 8 Stepping 6 (Celeron 700MHz)
>>FPU built-in
>>
>>
>>I do not suggest a CPU/FPU or Win98 problem, as I can add up the numbers
>>correctly using the calculator tool from the Win98 menue...?!
>>
>
>The calculator tool is probably using some sort of special library
>routines to add the big numbers up with. 'awk' uses the floating point
>unit in the processor, floating point works by having a certain number
>of bits for a mantissa (with x86 I think it's usually 53) and a certain
>number for an exponent. The actual value x will be represented as
>approximately 'mantissa + 2^exponent', so beyond a certain value you
[]
That should be 'mantissa * 2^exponent' (I avoid f.p like the plague <g>),
anyway the major point (fixed size, approximate representation) remains :-)
byefornow
--
laura fairhead # address@hidden http://lf.8k.com
# if you are bored crack my sig.
1F8B0808CABB793C0000666667002D8E410E83300C04EF91F2877D00CA138A7A
EAA98F30C494480157B623C4EF1B508FDED1CEFA9152A23DE35D661593C5318E
630C313CD701BE92E390563326EE17A3CA818F5266E4C2461547F1F5267659CA
8EE2092F76C329ED02CA430C5373CC62FF94BAC6210B36D9F9BC4AB53378D978
80F2978A1A6E5D6F5133B67B6113178DC1059526698AFE5C17A5187E7D930492