lynx-dev another largish patch

[Klaus Weide]

... is available temporarily at
   <http://enteract.com/~kweide/lynx/>


* Added support for ROWSPAN attribute of TD and TH to TRST.  This only
  reserves the appropriate amount of space in subsequent lines.

[ Pages that show off ROWSPAN nicely:
  <http://www.iro.umontreal.ca/contrib/po/HTML/domain-hello.html>
  and related ones for other "domain"s.  (Verify that there really
  is a ROWSPAN= in the current version.)  View with a screen width
  of 100.  With 80 there is some kind of problem, compare both 
  ^V settings... ]

* Some corrections in TRST code.

* Change initialization order of key escape sequence mappings for
  slang, so that terminfo/termcap information always overrides defaults
  that may conflict.

* Minimal memory clenup / leak prevention for EXP_FILE_UPLOAD error
  cases in GridText.c.
* In HText_SubmitForm, don't change the post_content_type of the
  passed in newdoc structure unless that is really wanted.

[ Untested. ]
[ Does the existing EXP_FILE_UPLOAD code work for anyone?  I mean
including the question whether the server understands the format. ]

* Tweaked UTF-8 prevention of display library wrapping/truncation
  to make better use of available width for centered text, and to
  not apply if dumping to stdout.
* For UTF-8 output with ncurses, do a clearok in display_page also
  if the page that was *previously* displayed had UTF-8 characters.
  Without this there were still display glitches.
* Made previous changes to SGML.c and HTMLDTD for handling OBJECT more
  generic.
* Minor tweaks, cleanups, glitch removal in previous changes.

* Better tracking of HTSprintf0/HTSprintf with --enable-find-leaks
  (-DLY_FIND_LEAKS).  This can be disabled in LYLeaks.h by
  removing the definition of LY_FIND_LEAKS_EXTENDED.
* Added variant behavior in StrAllocVsprintf to make HTSprintf0/HTSprintf
  use less memory allocation calls and keep its temporary string buffers
  across calls.  The buffers then only grow and never get cleaned up.
  This is enabled by defining SAVE_TIME_NOT_SPACE in HTString.c, disable
  the definition there in case of problems.
* Added code so HTSprintf0/HTSprintf can use vasprintf() in some
  situations, which should be more efficient.  It also works with
  --enable-find-leaks (EXTENDED or not). This only comes into play
  if USE_VASPRINTF is defined (not defined automatically anywhere).

[ For me (some Debian glibc version), the prototype for this function
is in stdio.h but only gets included with an explicit -D_GNU_SOURCE. ]
[ Which of the four combinations with/without vasprintf, with/without
SAVE_TIME_NOT_SPACE is better (if any) remains to be tested.  I have seen
drastic differences under leak tracking, but that's just too much
different from a normal situation to count for anything. ]

* Some minor leaks fixed. (LYMainLoop.c)

[ Had to make owner_address and ownerS_address file-wide PRIVATEs
 instead of auto to unleak them.  caould be taken out of more
 argument lists now. ]

* HTFile.c: correction in FormatNum (could duplicate fields for some
  LIST_FORMAT settings), avoid some HTSprintf0 calls in FormatStr.
* Replaced calls to HTSprintf with calls to HTSprintf0 in various files.
* Added missing include of LYLeaks.h in LYPrettySrc.c.  But disabled
  memory tracking there, too may allocations show up as leaks.

[ Vlad, you understand those structures, how about cleaning up the
memory... ]

* Allow digits in tagspec for -prettysrc.
* The 'fixit' change of 1999-05-16 didn't really let LYConvertToURL()
  revert to the right old behavior in the non-'g' use.  Now return a
  file URL for the nonexisting file in the relevant situation again,
  as before 1998-03-25, instead of an erroneous "file://localhost"
  which (on Unix-like systems at least) would result in access to the
  root directory.  This change (like the changes of 1999-05-16 and
  1998-03-25) only matters for strings that aren't already in absolute
  URL form, don't start with a slash (or, for DOSPATH systems, other
  path separator) either, and don't get turned into remote URLs by
  successful 'guessing' and DNS lookup.  None of the changes affect
  VMS.

* Protect INSERTFILE and EDITTEXTAREA from generating memory access
  violations if the inserted file has lines that are too long for
  the buffer used.  An alert message is issued and the long lines
  are truncated.  Also don't exit if memory allocation fails for
  the buffer to hold the whole file, since the only reason for the
  failure may be that the file is too huge when otherwise lynx has
  enough memory.
* Improved handling of some invalid constructs in SGML, especially
  '<' followed by various characters; don't lose those characters from
  what gets displayed for -prettysrc.
* A pre-filled TEXTAREA text that needed charset translation could cause
  use of an invalid pointer when building the page, resulting in garbage
  data being displayed (or worse).  The problem was in HTML.c, not related
  to TEXTAREA editing or other related new functions.

[ preceding three were partially already in previous patch ]

* Recognize '<?' as introducing an SGML PI.  Within most elements'
  content (or outside of any elements) this makes no effective difference 
  except for TRACE messages and -prettysrc, the input is still junked
  up to the next '>' as for a completely invalid tag.  Within PCDATA
  elements the PI is now also ignored instead of being treated as
  character data, this makes a difference for example in a TEXTAREA
  in SortaSGML mode.  (PI's aren't really expected to occur in the
  middle of HTML, but are used at the beginning of XHTML documents
  and other XML documents.)
* The earlier change of TEXTAREA handling (treating as SGML_PCDATA,
  SortaSGML only) created a problem, some lynx special characters were
  passed on as data characters in the field contents and would mess
  up line editing and possibly the submitted form data.  Actually the
  problem also existed before, but only for non break space, soft hyphen
  etc. encoded directly as character data (not NCRs or entities).
  Resolved by telling SGML.c not to generate lynx special chars for some
  element contents (currently only TEXTAREA), with new flag Tgf_nolyspcl.
* Corrected some logic errors in LYUCFullyTranslateString.  Special
  attribute characters are now more often translated to real characters
  when 'Back' flag is set, and should now be translated to the *right*
  characters for the target charset.  (They were sometimes converted
  to ISO-8859-1 values when the 'to' charset was different.)
* Added a missing significant cast in UCReverseTransChar.
* For scrollbar mouse clicking, use codes that work independent of current
  line-editor key bindings.
* Better calculation of "subjective distance" from next anchor for mouse.
  Change of 1999-07-30 had various problems, it wasn't quite intuitive.
  Use scaled numbers for finer granularity.  Don't make "basin of
  attraction" too wide, or it becomes hard to find a "background" spot
  to click on that doesn't affect a link if a page has a high density
  of links.

[ For example with a pair of fields:
       Submit Reset
              xxx
clicking in any of the positions marked 'x' on the next line would
select the first field, not the second. That was not an improvement. ]

* Functions LYmbcsstrlen, LYno_attr_mbcs_case_strstr, and
  LYno_attr_mbcs_strstr in LYStrings.c where unclear about the concept
  of 'printable' or 'physical' used for some returned values in the CJK
  case.  Callers expected number of display cells, implemented was number
  of 'characters' counting full-width characters as 1 instead of 2.
  Now these functions take an additional argument 'count_gcells' that
  tells them how to count.
  This removes some long-standing, somewhat obscure problems with search
  target highlighting under CJK display character sets, as well as new
  (and less obscure) CJK problems introduces by the latest changes for
  highlighting.  Also fixed some details in the *strstr functions.  The
  positioning of the highlighting in CJK mode should now be correct and
  work as expected (as for other display character sets) for anchors,
  search targets, and combinations of both.

[ Well, it should.  Now find the bugs please... ]

* Reorganized LYOpenTempRewrite.  It should finally make sense for systems
  without HAVE_TRUNCATE defined.

* Corrected SNACat (was unused).
* Changes for SUP and SUB: Render SUP visibly as a '^' character
  if it needs to be separated from a preceding character in order
  to prevent misinterpretation.  The test currently used is isxdigit
  for the the preceding character.  Render <SUB> and </SUB> always
  as brackets '[' and ']' respectively.  SUB is mostly used in
  technical material while SUP occurs frequently in pages of general
  nature where it is not essential, therefore the different treatment.

[ preceding changes were already in previous patch but not listed ]