Re: Mystery: Why does yylineno cause backing up tables?

[Hans Aberg]

At 11:41 -0800 2002/01/04, John W. Millaway wrote:
>> So maybe the important question should be: *why* is %%yylineno (claimed
>> to be by flex -p) such a performance penalty?
>
>Because it introduces REJECT code. When you set the option yylineno, flex
>behaves as if you had used REJECT in your actions. I don't understand why it
>must do this. As far as I can see, there are three places where yylineno must
>be adjusted:
>
>1. After a token has been matched.
>2. When the input buffer is manipulated via unput, yyless, or yymore.
>3. When REJECT is used.
>
>Nowhere should the adjustment of yylineo cause backing up, though. Correct?
>Perhaps Paxson could shed some light on this.

Perhaps he just put it in, because it an easy way to be on the safe side.

It seems me though that in the case you indicate, it should be easy to
recompute the line number. My thoughts, if I got the interpretation of
these rules correctly, assuming that the stream position file can always be
computed:
- At the start of the scan, one sets the begin and end location values to
the same value, the point where the scan begins.
- When a REJECT appears, if the end location has been set before the action
starts to execute, then the begin location value can be used to
re-initialize the end location value.
- When a yymore appears, at the next scan, the old begin location value is
used to set the new initial begin and end location value.
- When yyless(n) appears, then the skipped segments needs to be scanned for
newlines, to increment the count, and to find the position value of the
last newline.

Looking through these rule these rule, it strikes me that it might be the
case that the fact that one is keeping track of both the initial and end
location values simplifies, so that it is easier to reset to the correct
value: In the original Flex, my guess is that only _one_ line number is
kept track of, the current one. If a REJECT occurs, there is no value at
hand to reset to, unlike the situation with both a begin and end location
value.

  Hans Aberg