bug#56553: 29.0.50; ASAN error with fringe bitmaps on NS

[Gerd Möllmann]

> On 2022-07-14,, at 18:55 , Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> From: Gerd Möllmann <gerd.moellmann@gmail.com>
>> Date: Thu, 14 Jul 2022 16:34:05 +0200
>> Cc: 56553@debbugs.gnu.org
>> 
>> 
>> On 2022-07-14,, at 16:18 , Gerd Möllmann <gerd.moellmann@gmail.com> wrote:
>> 
>> frame #5: 0x000000010116d2b4 emacs`ns_define_fringe_bitmap(which=27,
>> bits=0x00000001066e1860, h=12, w=16) at nsterm.m:2906:20
>> 2903  /* XBM rows are always round numbers of bytes, with any unused
>> 2904  bits ignored. */
>> 2905  int byte = y * (w/8 + (w%8 ? 1 : 0)) + x/8;
>> -> 2906  bool bit = bits[byte] & (0x80 >> x%8);
>> 
>> I think the problem is indeed that bits is unsigned short*.  Otherwise, the 
>> /8 and %8 in line 2903 don't make
>> sense to me.  I think "byte" computes an index in a byte (char) array.
>> 
>> WDYT?
> 
> Do you understand what that loop is trying to do?  What is this line
> about:
> 
>        if (bit)
>          [p appendBezierPathWithRect:NSMakeRect (x, y, 1, 1)];
> 
> Anyway, it sounds like you are saying that the code wants to access
> the individual bytes of the 'short' elements?

The loop is in full

  for (int y = 0 ; y < h ; y++)
    for (int x = 0 ; x < w ; x++)
      {
        /* XBM rows are always round numbers of bytes, with any unused
           bits ignored.  */
        int byte = y * (w/8 + (w%8 ? 1 : 0)) + x/8;
        bool bit = bits[byte] & (0x80 >> x % 8);
        if (bit)
          [p appendBezierPathWithRect:NSMakeRect (x, y, 1, 1)];
      }

In pseudo-code, and with width and height replaced by actual values for the 
given case:

  for y from 0 to 12-1
    for x from ß to 16-1
      pixel = ...
      if pixel is set
        ...do some API stuff to remember how to draw it...

I think that is the intent.

If we insert the max values for x, y, w, h in the original, we have

   int byte = (12 - 1) * (16/8 + (16%8 != 0 ? 1 : 0) + (16 - 1)/8

which is 

                = 11 * (2 + (0 ? 1 : 0) + 15/8
                = 11 * 2 + 1
                = 23

which would be the right /byte/ to access in the bitmap.  But "bits[byte]" 
doesn't access the 23rd byte of the bitmap but the 23rd unsigned short, which 
is byte 46 and 47.  That cannot possibly be right, or?

The NSBezierPath stuff I don't know.  I gather, from a short look at the docs, 
that one can "record" stuff that should be drawn in such an object.  The path 
can then later be used to actually draw.  Looks a tad complicated to me to draw 
single pixels as a rectangle of size 1, but what do I know...