[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
token value return in custom yylex()
From: |
Bernd Prager |
Subject: |
token value return in custom yylex() |
Date: |
Tue, 3 Dec 2002 19:39:08 -0500 |
Hi,
I would appreciate some help, where I can't find the right info
in the manual.
In order to do some character escaping (and because my scanner is
really not that difficult) I wrote a custom yylex(). Everything seems to
be according to the manual, but the token value gets messed up in the
recursive grammar I use. (It works fine when I test my grammar with a flex.)
Can somebody tell me what I'm missing here?
Here is my code:
----------- snip --------------------
%{
#include <stdio.h>
#include <string.h>
#include "parser.tab.h"
#define TESTSTRING "word1 word2" // testpattern
char *data, buffer[1024];
%}
%union {
int id;
char *str;
}
%type <str> object token list
%token <id> WORD
%start object
%%
object: token
{
printf( "object: (token >%s<)\n", $1);
strcpy($$, $1);
}
| list
{
printf ("object: (list >%s<)\n", $1);
strcpy($$, $1);
}
;
/* a single word */
token: WORD
{
printf( "word: >%s<\n", $1);
strcpy($$, (char *)$1);
}
;
list: token object
{
char buf[1024];
printf( "term: (single >%s< object >%s<)\n", $1, $2);
sprintf($$, "%s %s", $1, $2);
}
;
%%
char *delimiters = "_,()' ";
char *lexbuffer;
char *clone;
char *yytext;
int length;
int main()
{
int i;
/* allocate memory for string lexbuffers */
length = strlen(TESTSTRING) + 1;
lexbuffer = (char *)malloc( length );
yytext = (char *)malloc( length );
strcpy( lexbuffer, TESTSTRING );
clone = lexbuffer;
i = yyparse();
free( lexbuffer );
free( yytext );
return i;
}
int yylex()
{
int i;
if (clone >= lexbuffer + length) return 0; // end of string
while (clone[0] == ' ' || clone[0] == '\t') clone++; // skip white space
if (strchr(delimiters, clone[0])!=NULL) { // character literal
i = clone[0];
clone++;
return i;
}
yylval.str = yytext;
for (i = 0; (i<length) && (strchr(delimiters, clone[0])==NULL); i++) {
if (clone[0] == '\\') { // escape next character
clone++;
}
yytext[i] = clone[0];
clone++;
}
yytext[i] = 0; // terminate yytext WORD
return WORD;
}
----------- snip --------------------
Here's the output with flex:
----------- snip --------------------
token: >word1<
token: >word2<
object: (token>word2<)
list: (token>word1< object >word2<)
object: (list>word1 word2<)
----------- snip --------------------
Here's mine:
----------- snip --------------------
token: >word1<
token: >word2<
object: (token >word2<)
list: (token >word2< object >word2<)
object: (list >word2 word2 <)
Segmentation fault
----------- snip --------------------
Thanks for any help.
-- Bernd