[Groff] preconv and hyphenation

[Werner LEMBERG]

Folks,


regarding our discussion about converting to Unicode I just want to
mention that this process basically disables hyphenation for
characters which aren't ASCII.  Unfortunately, this is an unavoidable
problem without a quick fix.

Reason is that all non-ASCII characters are converted to the \u[...]
form which no longer takes place in the hyphenation process.  This
problem will persist until GNU troff natively supports UTF-8 as input
encoding -- hyphenation *must* take place at the input character
level, and I won't repeat Knuth's error who made TeX apply hyphenation
at the glyph level, causing problems for languages which are
represented in more than a single font encoding (Russian, for
example).

With other words, the only people who will be completely happy about
preconv are Vietnamese because they don't use hyphenation at all :-)


    Werner