Re: confusing bison error

[Jot Dot]

> I'd prefer that you use the most recent release, although it shouldn't make 
> any difference here. 

I didn't think so either. I just prefer getting all the kinks out with the 
issues I have before introducing more variables such as changing a release. 
(That depends on what the release fixes, of course.) I do plan on staying 
current wrt new releases. 

> What do you mean? It did display "go to state 31", but did not? Could you 
> please provide us with the full trace? Also, have you tried to enable the 
> Flex traces, just in case. 

My input string is "CONST\n\trange = 1..10;\n" 
It picked up the identifier "range" and should have picked up the '=' (and then 
goto state 31) but says it go an error token instead. 

I don't know what you refer to in " enable the Flex traces ". 
Ok. So I can't get you a "reasonable" stack trace except for the attached .PNG 
file. 
The reason is apparent in the next section. I still have to write the code to 
save the generated debug output instead of just displaying it on the screen. 

Wow, you're brave :) 

Thought this was a "how it's done" thing these days. That's not the brave part. 
I've had issues trying to integrate the whole thing into my app in a coherent 
way. 
I got fed up and placed the whole scanner/parser into it's own DLL so to avoid 
any dependencies. 
I'm writing an MFC app which has a custom DLL loader, finds the DLL, loads it. 
There is a custom interface (virtual base class) where I communicate with the 
parser/scanner. 
I then start off the parser in it's own thread and redirect an iostream over 
pipes to communicate with the parser. 
This way a grammar change is just a load away. 
Talking to flex/bison is just a matter of using an iostream to feed it source 
and read back responses :) 

Now that's brave! 

fwiw: I'm tossing about the idea of throwing in this loader/flex-bison thing on 
github for others - once I get the kinks out. And properly documented of 
course. 

Can you show how you handle '='? 

'=' { return gen::Parser::symbol_type(*yytext, loc); } 

Hmm.. It probably has something to do with how I "pass through" the symbol so 
it can be used directly in the parser rule. 
The issue is probably here. Just not sure how if this is the correct way to 
pass the '-' back to the parser as a token with the same value. 
I'm not using the raw option for the parser. 

I learned to distrust Flex's handling of comments when they are in column 0. 

Yea, I'm starting to notice "something is up" with the first column. 

FWIW, I prefer to explicitly "continue;" in these cases. 

I'm a flex/bison newbie. Can you explain? (Not sure of "continue"). Any 
additional info is always welcome. 
I don't have any preferences since I just started with the idea of using 
something like flex/bison in my code. 
Before that, it was just some abstract concepts floating in my head. 
Well, not totally true. A couple years back I tried ANTLR but it was way too 
bug-ridden to be of any use. 
And the bug-report end was a bit uncaring and hostile imo. I wasted too much 
time with it. 

I don't understand either. You don't play with start conditions, do you? 

The only stuff is was I found on the web: 

%x SC_COMMENT 

%% 
int comment_nesting = 0; /* Line 4 */ 

"/*" { BEGIN(SC_COMMENT); } 
<SC_COMMENT>{ 
"/*" { ++comment_nesting; } 
"*"+"/" { if (comment_nesting) --comment_nesting; 
else BEGIN(INITIAL); } 
"*"+ ; /* Line 11 */ 
[^/*\n]+ ; /* Line 12 */ 
[/] ; /* Line 13 */ 
\n ; /* Line 14 */ 
} 


From: "Akim Demaille" <akim@lrde.epita.fr> 
To: "Jot Dot" <jotdot@shaw.ca> 
Cc: help-bison@gnu.org 
Sent: Friday, November 27, 2020 1:47:05 AM 
Subject: Re: confusing bison error 

Hi Jot, 


> Le 27 nov. 2020 à 01:11, Jot Dot <jotdot@shaw.ca> a écrit : 
> 
> FWIW: I'm new to flex/bison (lex/yacc) but I understand the basic concepts. 
> win_flex 2.6.4 
> bison (GNU Bison) 3.7.1 

I'd prefer that you use the most recent release, although it shouldn't make any 
difference here. 

> Having said that, everything compiles but it seems like the parser can't pick 
> up the single characters. 
> For example, the parser rule: 
> 
> single_const_decl: IDENTIFIER '=' expression ; 
> 
> The parser returns an error after it gets the IDENTIFIER and fetches an error 
> token instead of the '='. 
> I have figured out how to get the trace output and can confirm this: 
> 
> Trace says it goes into state 6: 
> 
> State 6 
> 
> 5 stmts: CONST_ . const_decl 
> 10 | CONST_ . const_decl ';' stmts 
> 46 const_decl: . single_const_decl 
> 47 | . const_decl ';' single_const_decl 
> 48 single_const_decl: . IDENTIFIER '=' expr 
> 
> IDENTIFIER shift, and go to state 14 <<<<< Trace says it goes to state 14 
> 
> const_decl go to state 15 
> single_const_decl go to state 16 
> 
> State 14 
> 
> 48 single_const_decl: IDENTIFIER . '=' expr 
> 
> '=' shift, and go to state 31 <<=== Does not happen. Returns an error token 
> instead. 

What do you mean? It did display "go to state 31", but did not? Could you 
please provide us with the full trace? Also, have you tried to enable the Flex 
traces, just in case. 


> FLEX: 
> 
> %option nodefault 
> %option c++ 
> %option yyclass="Scanner" 
> %option prefix="gen" 
> %option noyywrap 

Wow, you're brave :) 

> %option yylineno 
> %option case-insensitive 
> %option nounistd 
> %option stack 
> 
> inum [0-9]+ 
> alpha [A-Za-z_]+ 
> ident {alpha}({alpha}|{inum})* 
> 
> /* *snip* */ 
> 
> %% 
> 
> /* *snip* */ 
> {ident} { return gen::Parser::make_IDENTIFIER(result.makeIdentifier(yytext), 
> loc); } 

Can you show how you handle '='? 

> /* everything else */ 

I learned to distrust Flex's handling of comments when they are in column 0. 

> [ \t\r\n]+ ; /* whitespace */ 

FWIW, I prefer to explicitly "continue;" in these cases. 

> . { return gen::Parser::make_YYerror(loc); } // { yyerror("Unknown character 
> '%c'", *yytext); } 
> 
> 
> NOTE: In case this matters, I would have assumed the whitespace rule and the 
> '.' would have taken care of all possible input conditions but I still get 
> this: 
> 1>Target FlexTarget: 
> 1> Process "scanner.l" flex file 
> 1> scanner.l:154: warning, -s option given but default rule can be matched 

Wow, you're brave :) 

Can you show how you handle '='? 

I learned to distrust Flex's handling of comments when they are in column 0. 

FWIW, I prefer to explicitly "continue;" in these cases. 

I don't understand either. You don't play with start conditions, do you? 


Cheers!

stack trace.png
Description: PNG image