Re: LYNX-DEV Relative URLs, BASE implementation

[Foteos Macrides]

Klaus Weide <address@hidden> wrote:
>On Tue, 22 Oct 1996, Foteos Macrides wrote:
>[...]
>>      I suspect your assessment of not needing to understand the Lynx
>> code for mods of HTParse.c functions may be misguided (and your worsening
>> of the situation when you tried, perhaps should have been a tip off,
>> though a period of "trial and error" is important for anyone to get good
>> at supporting Lynx 8-).
>
>[...]
>  Actually, looking now (but still not looking closely) again at what I 
>did, I was calling the new lib's HTSimplify() from Lynx's lib's HTParse(),
>a not-so-bright idea alltogether (without really checking whether the 
>one was doing what the other expected).
>[...]

        The only standardized URL paramater is ;type=[A, D, I] for ftp,
and to my knowledge, Lynx is the only non-testbed browsers which recognizes
and acts on it.  Therefore, Lynx does it within the ftp gateway code, with
checks that it really is a typing parameter.  In the HTParse.c functions,
a ';' at the path has p < .00001 that it's the punctuation for a parameter
rather than a character actually part of the path that should have been
hex escaped.  It's a matter of judgement, but on today's Web, treating
anything but ;type= in ftp URLs as a parameter, with checks for that
really being what it appears, is likely to result in failure to fetch
the intended document.

        The handling of ?searchpart in the HTParse.c functions as distinct
from the path was disabled by Lou in conjunction with his implementation
of FORM support, when a ?searchpart could be either the query added to
an ISINDEX URI or the content for a FORM with METHOD=GET.  I wouldn't,
personally, have done it that way, but to change it would require
redesigns of the FORM handling, ISINDEX handling, gopher gateway, and
wais gateway, and no one has reported any problems in the "real world"
after all these years, so...

        There are some really useful things that could be done if
http URLs had some standardized parameters, but those things would
be Lynxisms unless the parameters were standardized and backward
compatibility issues were addressed.  My efforts to get standards
making groups interested in this haven't gotten anywhere.

        Dan Connolly of the W3C forwarded one of my messages about the
fragment parsing changes in RFC 1808 versus RFC 1738 to the IETF URI-WG
list, which is closed out but still maintained, and used terminology
which implies that if either the W3C or IETF undertook an update of
the URI/URL specs, that "cruft" should be reconsidered.  So in my
judgement we should stick with what RFC 1738 specified, which also
keeps Lynx compatible with the majority of currently deployed browsers.

        The decision whether to invoke the libwww-FM HTSimplify is made
in HTML.c.  That doesn't simplify gopher URLs, which are opaque by
definition, and it no longer simplifies absolute paths (beginning with
a slash) if they're going to a server (nor are lead "../" elements
stripped if there are more than would reach the host field on
simplification, which appears to have been a premature acceptance
of the RFC-based specs, and become an FAQ, sigh...).  Servers should
be set up themselves to avoid spoofing via relative specs.  Where it
might matter, is for file URLs, and HTML.c has quite a bit of code for
modifying the resolution of file URLs, and permits context dependent
decisions about simplification.  It seems better to keep that in
HTML.c than to do it indescriminately in the HTParse.c functions.

        Lynx uses only the HTTranslate() function in HTRules.c, which at
present is very small (has no effect on the image worth worrying about)
and doesn't change anything (need not be called).  However, redirection
the way Lou had originally designed it, was being done in HTMIME.c,
after commitments that caused unsolveable memory leaks, and unsolveable
security risks (e.g., Lynx could be redirected to URLs or actions that
were configured to be disallowed).  My mods are fine for HTTP/1.0
but have the disadvantage of ad hoc MIME header parsing in HTTP.c, and
sending you back to LYGetfile.c to, in effect, start all over again
with non-conventional security checks.  To make new HTTP/1.1 stuff really
work well, it might be better to use server-like rules and configuation
files, and more of the libwww5's code for access regulation and security
checks.  So I don't personally want to change things in the libwww-FM
that might promote others taking us further down a hacker's and perhaps
ultimately dead-end road.  Using server-like files means that things
like caching could be regulated properly, reliably, and based on whether
Lynx is being run on a strained multiuser system or powerful personal
computer.  You would want the select()-based threading for many of the
features you'd use on a powerful personal computer, and there's no
way to predict at this point how much performance would be affected.
If it were affected seriously, you might just as well have used a GUI
in the first place, with nicer screen management and font control.

                                Fote

=========================================================================
 Foteos Macrides            Worcester Foundation for Biomedical Research
 address@hidden         222 Maple Avenue, Shrewsbury, MA 01545
=========================================================================
;
; To UNSUBSCRIBE:  Send a mail message to address@hidden
;                  with "unsubscribe lynx-dev" (without the
;                  quotation marks) on a line by itself.
;