[Octave-bug-tracker] [bug #55682] round(X,N) and round(X,N,type)

[anonymous]

Follow-up Comment #12, bug #55682 (group octave):

There are some subtleties to the definition of round(X, N) relating to the
fact that many of the theoretical decimal rounding targets are not exactly
representable in binary floating point.  Consider:

x = 0.15;   % has actual value
0.1499999999999999944488848768742172978818416595458984375
round(x, 1) % what should this return?

The theoretical rounding targets of interest are 0.1 and 0.2, and their
corresponding nearest binary64 values are:

0.1000000000000000055511151231257827021181583404541015625
0.200000000000000011102230246251565404236316680908203125

Here, x would ideally round down regardless of whether we consider the
theoretical or the binary64 rounding targets.  Unfortunately the
implementation contemplated in earlier comments seems likely to round up in
this case.  I am not sure whether there are inputs for which the theoretical
and the binary64 rounding targets would imply different results.

Another consideration is that Octave does not specify the accuracy of the
exponentiation operator or the realpow function.  They do not seem to be
correctly rounded on my system.  For instance, `10 .^ -5 == 1e-5` is false. 
Perhaps things are different on the C++ side; I am not familiar with that.

Underflow and overflow could also cause problems.

Even in cases where the theoretical rounding targets are exactly
representable, setting aside issues of underflow and overflow, and assuming
exponentiation is correctly rounded, there is still potential for trouble: 
Computing `round(12345678.9, -5)` as `round(12345678.9 * 1e-5) / 1e-5` yields
12299999.99999999813735485076904296875 instead of 123e5.  The following might
be helpful if exponentiation can be relied upon to return an exact result when
it is representable:

   if (n > 0)
      retval = round(x .* 10 .^ n) ./ (10 .^ n);
   elseif (n < 0)
      retval = round(x ./ 10 .^ -n) .* (10 .^ -n);
   else
      retval = round(x);
   endif

I'm not certain that this never makes things worse though.



    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?55682>

_______________________________________________
Message sent via Savannah
https://savannah.gnu.org/