LYNX-DEV Character set support 4

[Michael Sokolov]

   Hi,
   
   I did get the latest developmental code (lynx2.7.1ac-0.23.zip), but the
problem with high control codes is still there. I actually agree with Klaus
on the idea that the raw mode shouldn't be absolutely raw. Klaus wrote that
the raw mode checks whether the terminal character set uses high control
codes and passes them unchanged if it does. I agree that this is the right
way, but that's not what the code currently does.
   I understand that you can't read Russian and your terminals don't have
the character set in question, so the page I have given you will always
look like garbage to you regardless of how it's rendered. However, since I
obviously do read Russian and have that character set on my terminal, I can
see that the page really IS rendered incorrectly. To verify my suspicion
that the problem is with high control codes, I have downloaded both the
original HTML and the Lynx output, loaded them in a decent DOS file viewer,
and compared them. And just as I expected, the "normal" 8-bit characters
(most lowercase Russian letters) displayed correctly, while the high
control codes (all uppercase Russian letters and some lowercase ones) were
either ignored or garbled.
   Actually, I know where the problem was in the release version 2.7.1. If
you remember, in my first message on this subject, I proposed a code change
to Lynx v2.7.1. Well, when I applied those changes to my copy of the
release version 2.7.1, it worked fine. In release 2.7.1, the function
HTMLSetCharacterHandling() in src/LYCharUtils.c contained a long if
statement with many "else if" clauses for setting special options for some
character sets. For character sets that use high control codes, it set
HTPassHighCtrlRaw to TRUE.
   Supposedly, in the new code the hard-coded tables and options in
src/LYCharSets.c and src/LYCharUtils.c are replaced with the more flexible
table-based mechanism. However, nowhere in the tables or their README*
files I saw a description of how to mark a character set as using high
control codes. Or maybe the old if statement in src/LYCharUtils.c is still
used?
   
   Sincerely,
   Michael Sokolov
   Phone: 216-646-1864
   ARPA Internet SMTP mail: address@hidden
;
; To UNSUBSCRIBE:  Send a mail message to address@hidden
;                  with "unsubscribe lynx-dev" (without the
;                  quotation marks) on a line by itself.
;