Re: Fw: Does flex support UTF-8

[Hans Aberg]

At 11:06 -0600 2004/03/30, Chuck Carmack wrote:
>I think we have a plan using utf-8 in our patterns that was suggested by
>Martin.  All we need to do is allow chars > 0x7f in an "identifier" (for
>example a,b,c, d below).  We have a utf-8 validator and a converter to
>utf-16 that we can have flex call (or have bison call... I'm a newbie on
>this).  We just need the raw bytes from flex for an identifier token.

I did not see this suggestion -- perhaps you forgot cc'ing the Flex list.

But in your tweaking, if you find a good way to implement UTF-8, please
report it back here, because I feel sure the Flex developers will be
interested. (I am not a Flex developer myself.)

But when working with Unicode, I figure that the idea must be to let Flex
be able to handle one Unicode encoding internally, gulping up raw bytes or
words. Then, if one needs another encoding in input, it is probably best to
hook up an external translator. So the trick will be to find which Unicode
encoding that works best with Flex. I think the most promising candidates
are UTF-8 and UTF24.

Also note that you should avoid styled text in the GNU mailing lists, as
not all readers may have it.

  Hans Aberg