Re: Buffer overrun when syntax fragment contains no terminating newline.

[John Darrington]

This fix seems to be causing test 1075 to fail.

On Sat, Sep 29, 2018 at 03:47:25PM -0700, Ben Pfaff wrote:
     On Sat, Sep 29, 2018 at 04:26:28PM +0200, John Darrington wrote:
     > I've just pushed a change fixing some sporadic crashes in the gui.   
     > 
     > The bug (which took a bit of tracking down) turned out to be caused by
     > a buffer overrun in lexer.c (lex_source_get_).   In particular, we have
     > the code:
     > 
     >  const char *newline = rawmemchr (line, '\n');
     > 
     > But the documentation for rawmemchr says that it's unpredictable if
     > line does not contain a '\n'.
     > 
     > So this means our syntax parser can crash if we present it with a
     > fragment which is not newline terminated.   I wasn't aware that we
     > had such a limitation.     Does this need to be fixed, or at least
     > explicitly documented ?
     
     Until recently, the lexer and its lower level infrastructure required
     source files to end in \n.  Because of this limitation, all the code
     that read source files added a trailing newline if one wasn't already
     present.  I fixed the limitation in commit e0f9210e814d ("lexer: Add
     support for embedded \0 bytes and missing trailing new-line.") because
     it made null bytes hard to handle properly.  At the same time, I removed
     the code to automatically add a trailing newline, because it was no
     longer necessary.
     
     In my code review, I missed this code that still assumed a trailing
     newline, and none of the tests caught it for me.  I pushed what I
     believe to be a fix now; I don't have enough time right at the moment to
     add some more tests, but I'll try to go back and add them later.

-- 
Avoid eavesdropping.  Send strong encrypted email.
PGP Public key ID: 1024D/2DE827B3 
fingerprint = 8797 A26D 0854 2EAB 0285  A290 8A67 719C 2DE8 27B3
See http://sks-keyservers.net or any PGP keyserver for public key.