Re: Graphics Rounding (was Re: Pixel-aligned autoresizing)

[Eric Wasylishen]

On 2011-07-11, at 1:23 AM, Riccardo Mottola wrote:

> Hi,
> 
>>> Sure. We have this code in NSButtonCell drawImage:withFrame:inView:
>>> 
>>> position.x = MAX(NSMidX(cellFrame) - (size.width / 2.), 0.);
>>> position.y = MAX(NSMidY(cellFrame) - (size.height / 2.), 0.);
>>> 
>>> Suppose position.x is 16.5, then rint() would round that to 16, so the 
>>> image will be drawn half a pixel to the left of where it should be.
>>> 
>>> The problem is if we move the button one point to the right, position.x 
>>> will be 17.5, and rint() rounds that to 18, so suddenly the image is drawn 
>>> right of where it should be.  You wouldn't expect moving the button frame 
>>> to affect the position of the image inside, but it does. i.e. whether 
>>> something is at an even or odd coordinate should be unimportant for 
>>> graphics, but rint() decides which way to round based on that.
> 
> To my knowledge, rint() doesn't do any advanced rounding. It doesn't try to 
> minimize the error by rounding odd numbers down and even up or something like 
> that, it uses a standard round direction.  16.5 will become 17 as 17.5 will 
> become 18. That is, the round error hasn't a zero expected value.
> 
> 
> To prove that, I executed a loop in 0.1 steps and printed out the results:
> 
> long float: 16.000000, rounded: 16.000000
> long float: 16.100000, rounded: 16.000000
> long float: 16.200000, rounded: 16.000000
> long float: 16.300000, rounded: 16.000000
> long float: 16.400000, rounded: 16.000000
> long float: 16.500000, rounded: 17.000000
> long float: 16.600000, rounded: 17.000000
> long float: 16.700000, rounded: 17.000000
> long float: 16.800000, rounded: 17.000000
> long float: 16.900000, rounded: 17.000000
> long float: 17.000000, rounded: 17.000000
> long float: 17.100000, rounded: 17.000000
> long float: 17.200000, rounded: 17.000000
> long float: 17.300000, rounded: 17.000000
> long float: 17.400000, rounded: 17.000000
> long float: 17.500000, rounded: 18.000000
> long float: 17.600000, rounded: 18.000000
> long float: 17.700000, rounded: 18.000000
> long float: 17.800000, rounded: 18.000000
> long float: 17.900000, rounded: 18.000000
> 
> 
> As you see, it is consistent. rint() man page says on linux:
> 
>       The nearbyint() functions round their argument to an integer  value  in
>       floating-point  format,  using the current rounding direction (see fesâ
>       etround(3)) and without raising the inexact exception.
> 
> 
> rint() does the same. The point is "using the current rounding direction".

Yeah - the problem is the specification is very loose. It would be compliant
with c99 to implement rint as floor(), ceil(), round(), or to use any of the
tie-breaking algorithms for 0.5 cases.

c libraries are allowed to let the user change the rounding mode using 
fsetround, but there are no required rounding modes.

I think the reason for the round-to-even behaviour showing up for me 
is it is the default mode specified by IEEE754:
http://en.wikipedia.org/wiki/IEEE_754-2008#Rounding_algorithms

so it's probably implemented in hardware. The design of rint seems to be
so C libraries can implement it using a CPU instruction without caring
what the rounding algorithm actually is.

> I don't know if the rint() behaviour is consistent among math libraries.

It doesn't seem to be consistent. Here is what I get on my Ubuntu system

new-host-2:gui ericw$ cat ~/test.c
#include <math.h>
#include <stdio.h>

int main()
{
        double a;
        for (a=0.5; a<4; a+= 0.5)
        {
                printf("round(%g)=%g rint(%g)=%g\n", a, round(a), a, rint(a));
        }
        return 0;
}


address@hidden:~$ gcc -lm test.c
address@hidden:~$ ./a.out 
round(0.5)=1 rint(0.5)=0
round(1)=1 rint(1)=1
round(1.5)=2 rint(1.5)=2
round(2)=2 rint(2)=2
round(2.5)=3 rint(2.5)=2
round(3)=3 rint(3)=3
round(3.5)=4 rint(3.5)=4
address@hidden:~$ uname -a
Linux ericwa-VirtualBox 2.6.35-22-generic #33-Ubuntu SMP Sun Sep 19 20:32:27 
UTC 2010 x86_64 GNU/Linux

And my Macbook:

new-host-2:~ ericw$ ./a.out 
round(0.5)=1 rint(0.5)=0
round(1)=1 rint(1)=1
round(1.5)=2 rint(1.5)=2
round(2)=2 rint(2)=2
round(2.5)=3 rint(2.5)=2
round(3)=3 rint(3)=3
round(3.5)=4 rint(3.5)=4
new-host-2:~ ericw$ uname -a
Darwin new-host-2.home 10.8.0 Darwin Kernel Version 10.8.0: Tue Jun  7 16:33:36 
PDT 2011; root:xnu-1504.15.3~1/RELEASE_I386 i386


So I get the round-to-even behaviour on both systems - 2.5 rounds to 2, 3.5 
rounds to 4.

Anyway, I think the fact that there are no guarantees about what it will do is 
a good reason to avoid it for graphics, where cpu time for rounding is of no 
concern but output quality is.

--Eric