Re: WWWLib 5 in LYNX-DEV Lynx 3.0

[Klaus Weide]

On Wed, 26 Feb 1997, Hiram Lester, Jr. wrote:
> On Wed, 26 Feb 1997, T.E.Dickey wrote:
> 
> > > One project that I've been meaning to do is to convert the filenames to
> > > the 8.3 convention (at least in the WWWLib) to make the job of integrating
> > > WWWLib 5 into the code a little easier.  Has Mr. Dickey (sorry, brain
> > > isn't working, and I can't remember your first name) taken a look
at the
> > > autoconfigure script for WWWLib 5 and see if anything can be done to it to
> > > remove its dependence on GNU Make?
> > I've not looked at it - where should I look?
> 
> WWWLib is on W3C's site: www.w3.org

And, to be slighty more specific, for getting the libwww sources and
docs:  <URL: http://www.w3.org/pub/WWW/Library/>.

> You can get there by following the link in Lynx's help files, or from the
> index page if you haven't changed it from the default at uiuc.
> 
> On of the major problems with WWWLib 5 (other than the fact that some
> existing stuff in Lynx's WWWLib would have to either be re-incorporated
> into the new code or adapted to the new code's way of doing things) is
> that it relies on GNU Make.  This would be fine for Linux and *BSD boxes,
> but most other platforms would be required to acquire GNU Make to build a
> Lynx built with WWWLib 5... :(

The libwww code has to be massaged so much anyway that you cannot hope to
incorporate it as-is, installation procedures and all, into the Lynx
code...  The code itself doesn't depend on a specific Make any more than
the Lynx code - I think.  You just have to get all thos necessary symbols
defined and headers included etc., But there is no reason why you have to
absolutely *have* to get that done with the W3C's autoconf/GNU Make
procedure.

> Klaus has done some work on incorporating WWWLib 5 into an earlier version
> of lynx (2-5FM?), 

No, 2-6, with a few of the later changes added in...

> so some of that may still be useful. 

I am still playing with it.  I have a running Lynx which uses the libwww 5.0A
Protocol modules (at least for HTTP).  I am actually using it.  It
currently doesn't do POSTing, and it doesn't do Authentication.  But a
plain old GET request works just fine - well most of the time.
It is able to keep persistent HTTP connections.  But if you interrupt a
retrieval, strange things happen.

I spent a lot of time getting the disk cache to work.  It does work
somewhat.  Well on the other hand last year I have argued (together with
Fote) that Lynx doesn't really need a disk cache (mostly because there are
external caching proxies that can be installed, which probably do a better
job of caching than Lynx could ever do.  because that is all they do.)
I still think Lynx doesn't really need one...

I also spent a lot of time figuring out how to handle a little thing like
a HEAD request (Lynx's ']' key).  The library was not ready for such
tasks, at least not when talking to a HTTP 1.1 server.  (Or I have not
found the obvious way to do it.)  In general, the library code doesn't
have the kind of maturity that nearly everything in the Lynx code has.
(It just hasn't been tested and debugged by nearly as many people, is one
thing.)  While it may work under normal conditions, it isn't prepared to
deal well with exceptional things (user interrupt, unexpected server
responses etc.).

If anybody wants to see the current mess, drop me a note...
It may not work on anything but Linux, I haven't tried to build it
anywhere else.

Unfortunately that code has diverged a lot from the mainstream Lynx code,
and merging the changes (or, more likely, parts of them) will be a pain.
Of course I failed to save any snapshots in the beginning when the changes
were still less extensive...  Still, it could be worse.  I think most of
the changes since 2.6 were in other areas than those I was dealing with.

I think now that the most realistic approach for putting some of this back
into a public Lynx version is to start with some small (but extensive) 
changes - basically starting to redo the whole thing.  Like introducing
some new data structures (HTRequest, HTNet, HTResponse, ...) used
throughout the libwww5 code, changing all HTStream methods to return a
result everywhere in the Lynx code.  Those would be changes that would, 
at first, not change anything in what the code actually does.  But it
would then be easier to add other pieces; or grab parts from the existing
libwww5 and use them (maybe the DNS lookup cache...)

> All I did was pull
> down the source and see if I could compile it on HP-UX's make. (I
> couldn't)
> 
> Since you seem to be our resident autoconf expert :) I thought I'd mention
> it.

Starting point for my libwww5 hacking was actually the stuff to which I
pointed T.E.D. (at <URL:http://www.tezcat.com/~kweide/lynxhacks/>); which
should theoretically also work for mainstream Lynx..

I would find it helpful if the people working on auto-configuration for
Lynx could have the libwww5 configuration in mind.  Maybe just, when you
have to introduce a new preprocessor symbol, first check whether the
libwww5 sysdep.h or Makefile or config.h already has it, and then use the
same name rather than inventing a new one.


  Klaus

;
; To UNSUBSCRIBE:  Send a mail message to address@hidden
;                  with "unsubscribe lynx-dev" (without the
;                  quotation marks) on a line by itself.
;