gnustep-dev
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Floating point rounding errors in NSTableView?


From: David Chisnall
Subject: Re: Floating point rounding errors in NSTableView?
Date: Tue, 30 Jun 2015 12:13:21 +0100

On 30 Jun 2015, at 10:43, Matt Rice <address@hidden> wrote:
> 
> is it apparent in the return value of -[NSTableView rectOfRow:] for
> the afflicted rows?

Yes, it does seem to be.  I added this code to run when it’s finished preparing 
all of the rows:

        CGFloat lastY = 0;
        for (int i=1 ; i<100 ; i++)
        {
            auto r = [traceView rectOfRow:loadedEntries-i];
            NSLog(@"Rect for row %lld: %@ (%lf)", loadedEntries-i, 
NSStringFromRect(r),r.origin.y - lastY);
            lastY = r.origin.y;
        }

The output looks like this (lastY is nonsense for the first line - ignore it).

2015-06-30 10:58:24.075 CheriVis[16718] Rect for row 123832861: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (2105158656.000000)
2015-06-30 10:58:24.075 CheriVis[16718] Rect for row 123832860: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.075 CheriVis[16718] Rect for row 123832859: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (-128.000000)
2015-06-30 10:58:24.075 CheriVis[16718] Rect for row 123832858: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.075 CheriVis[16718] Rect for row 123832857: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.075 CheriVis[16718] Rect for row 123832856: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.076 CheriVis[16718] Rect for row 123832855: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.076 CheriVis[16718] Rect for row 123832854: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.076 CheriVis[16718] Rect for row 123832853: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.076 CheriVis[16718] Rect for row 123832852: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (-128.000000)
2015-06-30 10:58:24.076 CheriVis[16718] Rect for row 123832851: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.076 CheriVis[16718] Rect for row 123832850: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.076 CheriVis[16718] Rect for row 123832849: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.076 CheriVis[16718] Rect for row 123832848: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.076 CheriVis[16718] Rect for row 123832847: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.076 CheriVis[16718] Rect for row 123832846: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.076 CheriVis[16718] Rect for row 123832845: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.076 CheriVis[16718] Rect for row 123832844: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.076 CheriVis[16718] Rect for row 123832843: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (-128.000000)
2015-06-30 10:58:24.076 CheriVis[16718] Rect for row 123832842: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.076 CheriVis[16718] Rect for row 123832841: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.076 CheriVis[16718] Rect for row 123832840: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.076 CheriVis[16718] Rect for row 123832839: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.076 CheriVis[16718] Rect for row 123832838: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.076 CheriVis[16718] Rect for row 123832837: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.076 CheriVis[16718] Rect for row 123832836: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (-128.000000)
2015-06-30 10:58:24.076 CheriVis[16718] Rect for row 123832835: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.076 CheriVis[16718] Rect for row 123832834: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.076 CheriVis[16718] Rect for row 123832833: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.076 CheriVis[16718] Rect for row 123832832: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.076 CheriVis[16718] Rect for row 123832831: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.076 CheriVis[16718] Rect for row 123832830: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.076 CheriVis[16718] Rect for row 123832829: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.077 CheriVis[16718] Rect for row 123832828: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.077 CheriVis[16718] Rect for row 123832827: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (-128.000000)
2015-06-30 10:58:24.077 CheriVis[16718] Rect for row 123832826: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.077 CheriVis[16718] Rect for row 123832825: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.077 CheriVis[16718] Rect for row 123832824: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.077 CheriVis[16718] Rect for row 123832823: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.077 CheriVis[16718] Rect for row 123832822: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.077 CheriVis[16718] Rect for row 123832821: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.077 CheriVis[16718] Rect for row 123832820: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (-128.000000)
2015-06-30 10:58:24.077 CheriVis[16718] Rect for row 123832819: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.077 CheriVis[16718] Rect for row 123832818: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.077 CheriVis[16718] Rect for row 123832817: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.077 CheriVis[16718] Rect for row 123832816: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.077 CheriVis[16718] Rect for row 123832815: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.077 CheriVis[16718] Rect for row 123832814: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.077 CheriVis[16718] Rect for row 123832813: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.077 CheriVis[16718] Rect for row 123832812: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.077 CheriVis[16718] Rect for row 123832811: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (-128.000000)
2015-06-30 10:58:24.077 CheriVis[16718] Rect for row 123832810: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.077 CheriVis[16718] Rect for row 123832809: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.077 CheriVis[16718] Rect for row 123832808: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.077 CheriVis[16718] Rect for row 123832807: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.077 CheriVis[16718] Rect for row 123832806: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.077 CheriVis[16718] Rect for row 123832805: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.077 CheriVis[16718] Rect for row 123832804: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (-128.000000)
2015-06-30 10:58:24.077 CheriVis[16718] Rect for row 123832803: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.077 CheriVis[16718] Rect for row 123832802: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.078 CheriVis[16718] Rect for row 123832801: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.078 CheriVis[16718] Rect for row 123832800: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.078 CheriVis[16718] Rect for row 123832799: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.078 CheriVis[16718] Rect for row 123832798: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.078 CheriVis[16718] Rect for row 123832797: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.078 CheriVis[16718] Rect for row 123832796: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.078 CheriVis[16718] Rect for row 123832795: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (-128.000000)
2015-06-30 10:58:24.078 CheriVis[16718] Rect for row 123832794: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.078 CheriVis[16718] Rect for row 123832793: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.078 CheriVis[16718] Rect for row 123832792: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.078 CheriVis[16718] Rect for row 123832791: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.078 CheriVis[16718] Rect for row 123832790: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.078 CheriVis[16718] Rect for row 123832789: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.078 CheriVis[16718] Rect for row 123832788: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (-128.000000)
2015-06-30 10:58:24.078 CheriVis[16718] Rect for row 123832787: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.078 CheriVis[16718] Rect for row 123832786: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.078 CheriVis[16718] Rect for row 123832785: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.078 CheriVis[16718] Rect for row 123832784: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.078 CheriVis[16718] Rect for row 123832783: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.078 CheriVis[16718] Rect for row 123832782: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.078 CheriVis[16718] Rect for row 123832781: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.078 CheriVis[16718] Rect for row 123832780: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.078 CheriVis[16718] Rect for row 123832779: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (-128.000000)
2015-06-30 10:58:24.078 CheriVis[16718] Rect for row 123832778: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.078 CheriVis[16718] Rect for row 123832777: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.078 CheriVis[16718] Rect for row 123832776: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.078 CheriVis[16718] Rect for row 123832775: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.078 CheriVis[16718] Rect for row 123832774: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.079 CheriVis[16718] Rect for row 123832773: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.079 CheriVis[16718] Rect for row 123832772: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (-128.000000)
2015-06-30 10:58:24.079 CheriVis[16718] Rect for row 123832771: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.079 CheriVis[16718] Rect for row 123832770: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.079 CheriVis[16718] Rect for row 123832769: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.079 CheriVis[16718] Rect for row 123832768: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.079 CheriVis[16718] Rect for row 123832767: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.079 CheriVis[16718] Rect for row 123832766: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.079 CheriVis[16718] Rect for row 123832765: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.079 CheriVis[16718] Rect for row 123832764: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (0.000000)
2015-06-30 10:58:24.079 CheriVis[16718] Rect for row 123832763: {x = 0; y = 
2.10516e+09; width = 702; height = 17} (-256.000000)

This corresponds to roughly what I see.  A load of lines drawn on top of each 
other, and then a big gap.  

> _rowHeight contributing to the rect.origin.y seems
> to be a float rather than CGFloat?

Indeed.  Changing it to CGFloat fixes it for me (at least on 64-bit platforms, 
though I suspect that it will break on 32-bit platforms?).  I wonder if the 
columnWidths array also needs changing (I’ve never tried an NSTableView with 
millions of columns, but I suspect that it’s not very scalable in that 
direction anyway)?

David

-- Sent from my STANTEC-ZEBRA




reply via email to

[Prev in Thread] Current Thread [Next in Thread]