[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Reserved Words
From: |
Mike Aubury |
Subject: |
Re: Reserved Words |
Date: |
Thu, 20 Feb 2003 18:37:19 +0000 |
I've replaced 'a' with A, 'b' with B...
All my tokens are tokenised (no '.'), and uppercased in the grammer....
In my checks - if I get a -1 - that is always passed through unchanged ($end
can't really be an identifier!)
>From my script - we get :
state 0 accepting :
A
state 1 accepting :
A or B
state 2,4,5 accepting nothing at all - I don't think yylex would be called in
these states...
state 3 accepting
B
(Which I think is right!)
It may be pure chance that it works - but I'm reasonably happy that it
does....
On Thursday 20 February 2003 4:38 pm, Akim Demaille wrote:
> Mike> Cheers for that - I actually managed to do it a different
> Mike> way... Using bison 1.875 - there is a -r all option - I used
> Mike> this to create a y.output which (helpfully) descends states
> Mike> (unlike some older versions..) and prints all the possible
> Mike> tokens at each state..
>
> Actually, it doesn't... It reports *needed* lookaheads. Consider the
>
> following:
> | Grammar
> |
> | 0 $accept: e $end
> |
> | 1 e: a 'b'
> | 2 | 'a' 'a'
> |
> | 3 a: 'a'
> |
> |
> | Terminals, with rules where they appear
> |
> | $end (0) 0
> | 'a' (97) 2 3
> | 'b' (98) 1
> | error (256)
> |
> |
> | Nonterminals, with rules where they appear
> |
> | $accept (5)
> | on left: 0
> | e (6)
> | on left: 1 2, on right: 0
> | a (7)
> | on left: 3, on right: 1
> |
> |
> | state 0
> |
> | 0 $accept: . e $end
> | 1 e: . a 'b'
> | 2 | . 'a' 'a'
> | 3 a: . 'a'
>
> It should have written [$end] for 1 and 2, and ['b'] for 3.
>
> | 'a' shift, and go to state 1
> |
> | e go to state 2
> | a go to state 3
> |
> |
> | state 1
> |
> | 2 e: 'a' . 'a'
> | 3 a: 'a' . ['b']
>
> Missing [$end] for 2. But it just needs to know for ['b'] to make its
> mind here, so it did compute the lookahead and reports it.
>
> | 'a' shift, and go to state 4
> |
> | $default reduce using rule 3 (a)
> |
> |
> | state 2
> |
> | 0 $accept: e . $end
> |
> | $end shift, and go to state 5
> |
> |
> | state 3
> |
> | 1 e: a . 'b'
>
> Missing [$end] etc.
>
> | 'b' shift, and go to state 6
> |
> |
> | state 4
> |
> | 2 e: 'a' 'a' .
> |
> | $default reduce using rule 2 (e)
> |
> |
> | state 5
> |
> | 0 $accept: e $end .
> |
> | $default accept
> |
> |
> | state 6
> |
> | 1 e: a 'b' .
> |
> | $default reduce using rule 1 (e)
>
> Mike> Using a %pure_parser - I pass that state into my yylex - and
> Mike> then check to see if the token is allowed in that state...
>
> Amazing! This is probably very fragile, no? I mean, does it depend
> on internals?
>
> Mike> If it is - then I let it through, if its not - the I convert it
> Mike> to the identifier token (after checking it looks like an
> Mike> identifier a-z etc)...
>
> Mike> Seems to work like a charm... (Over 500,000 lines of our
> Mike> language source code correctly parsed as a test).
>
> Congrats!