Re: why are locations dictated by bison?

[Hans Aberg]

At 13:36 +0100 2002/01/12, Akim Demaille wrote:
>.           {
>  yylloc->word (yyleng);
>  std::cerr << *yylloc << ": invalid character: `"
>           << *yytext << "'" << std::endl;
>  if (!exit_status)
>    exit_status = exit_scan;
>  }

I tweaked the Bison skeleton file (see the Bug Bison list) so that this
error message now is written out by the Bison parser. Thus, instead write
  <INITIAL>.  { return (unsigned char)yytext[0]; }
and the parser will tell that an undefined character was encountered. (This
is possible becaue %raw was removed from Bison.)

Bison reserves the values 0 (initial/end "$") and 1 ("error"), but the
other non-character tokens have values > 256, so one knows that tokens in
the range
2...255 must be character tokens. It would have been better if 0 and 1
would not have been used by Bison, because the \0 and \1 would have worked
in thois scheme. For Unicode, I think the parser token should start at 2^N
+ 1, where perhaps N = 24 (N = 20-21 reserved for Unicode characters and
the rest for "user characters").

I also found it useful to let the lexer return YYERRCODE, thus giving the
parser the responsibility to handle errors, which can be subjected to error
recovery then. I think this macro YYERRCODE (as suggested on the Bug Bison
list) should be written into the <name>.tab.h header, so that one needs not
write it out in the lexer.

  Hans Aberg