Re: Parsing a language with optional spaces

[John P. Hartmann]

On 7/7/20 05:35, Akim Demaille wrote:
> I believe you need to read again the documentation of /
>
> 'r/s'

It is not as simple as that.  As I don't speak BASIC, let me rephrase 
this problem in FORTRAN IV which is also "blank agnostic":

DO <number> <variable> = <expression> , <expression> [, <expression>]

It is not until you reach the comma after the first expression that you 
know whether the statement is the beginning of a loop or it is an 
assignment.  And the expression can contain commas in function calls, 
which defeats any trivial lookahead scanning.  E.g.,

D O 17 6PQ R=FUN X(1 4, V 8)

is an assignment to variable DO176PQR.  The function arguments can also 
be expressions that contain function calls.

As you can see, this more or less defeats any attempt to write a lex 
scanner.  And you cannot just squeeze out all blanks in a front end 
because "Hollerith fields" can contain blanks that are significant (must 
remain).