Two problems with the GLR parsers

[Akim Demaille]

Hi Paul,

There are two issues which are uncovered by the test suite:

   42. actions.at:368: testing Printers and Destructors: %glr-parser...
   actions.at:368: bison -o input.c input.y
   actions.at:368: $CC $CFLAGS $CPPFLAGS input.c -o input
   stderr:
   actions.at:368: $PREPARSER ./input
   --- -   2002-11-12 11:16:02.000000000 +0100
   +++ /home/lrde/prof/akim/src/bison/tests/testsuite.dir/at-stdout        
2002-11-12 11:16:02.000000000 +0100
   @@ -15,7 +15,7 @@
    sending: 'x' (value = 5, line 50)
    Freeing token 'x' (address@hidden)
    sending: ';' (value = 6, line 60)
   -line(address@hidden): error(@50) ';'
   +line(address@hidden): error(@30) ';'
    sending: 'x' (value = 7, line 70)
    thing(address@hidden): 'x'(address@hidden)
    sending: 'x' (value = 8, line 80)
   @@ -31,6 +31,6 @@
    sending: EOF
    Freeing nterm line (address@hidden)
    Freeing nterm line (address@hidden)
   -Freeing nterm line (address@hidden)
   -Parsing FAILED.
   +Freeing nterm line (address@hidden)
   +Successful parse.
   
   actions.at:368: exit code was 0, expected 1
   42. actions.at:368: FAILED near `actions.at:368'
   

One problem is that for unknown reasons, it is not the same location
that is associated to the error token as the one from yacc.c.  This
might be completely benign, I have not really tracked it down.  This
symbol comes from

| error ';'
    {
      $$ = -1;
      printf ("line(address@hidden): error(@%d) ';'\n",
              $$, @$.first_line, @1.first_line);
    }

I have not looked at it in details yet, maybe there is no reason to
waste time on this.


But the second issue is a bit different: the parse exits with success
although it actually failed.  The problem is that 

   /* Recover from a syntax error on YYSTACK, assuming that YYTOKENP,
      YYLVALP, and YYLLOCP point to the syntactic category, semantic
      value, and location of the lookahead.  */
   static void
   yyrecoverParseError (yyGLRStack* yystack,
                        YYSTYPE* yylvalp, YYLTYPE* yyllocp]b4_user_formals[)
   {
     yySymbol* const yytokenp = yystack->yytokenp;
     size_t yyk;
     int yyj;
   
     if (yystack->yyerrState == 0)
       yystack->yyerrState = 3;
     else if (yystack->yyerrState == 3)
       /* We just shifted the error token and (perhaps) took some
          reductions.  Skip tokens until we can proceed.  */
       while (yytrue)
         {
           if (*yytokenp == YYEOF)
             {
               /* Now pop stack until we find a state that shifts the
                  error token.  */
               while (yystack->yytops.yystates[0] != NULL)
                 {
                   yyGLRState *yys = yystack->yytops.yystates[0];
                   YYDPRINTF ((stderr, "Error: popping "));
                   YYDSYMPRINT ((stderr,
                                 yystos[yys->yylrState],
                                 &yys->yysemantics.yysval]b4_location_if([, 
&yys->yyloc])[));
                   YYDPRINTF ((stderr, "\n"));
                   yydestruct (yystos[yys->yylrState],
                               &yys->yysemantics.yysval]b4_location_if([, 
&yys->yyloc])[);
                   yystack->yytops.yystates[0] = yys->yypred;
                   yystack->yynextFree -= 1;
                   yystack->yyspaceLeft += 1;
                 }
               yyFail (yystack][]b4_lpure_args[, NULL);
             }

we arrive here after having called yyerror via yyreportParseError.
yyerror does not set errflag, as it is yyFail that calls yyerror, not
the converse.

I see several means to fix this issue, but I am not sure how you'd do
it.  One obvious fix is to add yystack->errflag = 1 right before the
yyFail call above.  Another is to have yyFail always do it, included
when called with NULL.  But I'm not sure of the semantics of yyFail :(

Also, yyparse looks like this:


   /*----------.
   | yyparse.  |
   `----------*/
   
   int
   yyparse (void)
   {
   ///.....
        while (yytrue)
           {
             int yys;
             int yyn = yystack.yytops.yysize;
             for (yys = 0; yys < yyn; yys += 1)
               YYCHK1 (yyprocessOneStack (&yystack, yys, yyposn,
                                          yylvalp, yyllocp));
             yytoken = YYEMPTY;
             yyposn += 1;
             yyremoveDeletes (&yystack);
             if (yystack.yytops.yysize == 0)
               {
                 yyundeleteLastStack (&yystack);
                 if (yystack.yytops.yysize == 0)
   =====>          yyFail (&yystack, "parse error");
                 YYCHK1 (yyresolveStack (&yystack));
                 YYDPRINTF ((stderr, "Returning to deterministic 
operation.\n"));
                 yyreportParseError (&yystack, yylvalp, yyllocp);
                 goto yyuser_error;
               }
             else if (yystack.yytops.yysize == 1)
               {
                 YYCHK1 (yyresolveStack (&yystack));
                 YYDPRINTF ((stderr, "Returning to deterministic 
operation.\n"));
                 yycompressStack (&yystack);
                 break;
               }
           }
         continue;
       yyuser_error:
         yyrecoverParseError (&yystack, yylvalp, yyllocp);
         yyposn = yystack.yytops.yystates[0]->yyposn;
       }
    yyDone:
     ;
   
     yyfreeGLRStack (&yystack);
     return yystack.yyerrflag;
   }


I don't know why this line is not using yyreportParseError.  Actually,
I don't know what yystack.yytops.yysize == 0 means :(