Re: WWWLib 5 in LYNX-DEV Lynx 3.0

[Klaus Weide]

On Thu, 27 Feb 1997, Hiram Lester, Jr. wrote:
> On Wed, 26 Feb 1997, Klaus Weide wrote:
> 
> > On Wed, 26 Feb 1997, Hiram Lester, Jr. wrote:
> >
> > > On of the major problems with WWWLib 5 (other than the fact that some
> > > existing stuff in Lynx's WWWLib would have to either be re-incorporated
> > > into the new code or adapted to the new code's way of doing things) is
> > > that it relies on GNU Make.  This would be fine for Linux and *BSD boxes,
> > > but most other platforms would be required to acquire GNU Make to build a
> > > Lynx built with WWWLib 5... :(
> > 
> > The libwww code has to be massaged so much anyway that you cannot hope to
> > incorporate it as-is, installation procedures and all, into the Lynx
> > code...  The code itself doesn't depend on a specific Make any more than
> > the Lynx code - I think.  You just have to get all thos necessary symbols
> > defined and headers included etc., But there is no reason why you have to
> > absolutely *have* to get that done with the W3C's autoconf/GNU Make
> > procedure.
> 
> Go back and look at the docs for it again.  It specifically states that it
> requires GNU Make.  

Yes, it says

             - Build the library using GNU Autoconf's configure script.
                   Note: You must use GNU make in order to do this

etc. on <URL: http://www.w3.org/pub/WWW/Library/User/Platform/>.  My
understanding is that that is for people who want to download the libwww
code and then have an installation preocedure which builds the library
automatically.  But it wouldn't be relevant any more once (parts of) the
new library code have been integrated into the Lynx distribution.  That
code would then be subject to Lynx's configuration/installation mechanisms
(whatever they will be at that point).

Of course it helps, for starting to play with it, if you can create the
WWW Library with its out-of-the-box installation scripts.  But it is not a
requirement for the final consumer - the user and the machine where Lynx
code has to be installed and run.

> I tried compiling it on HP-UX, and it got errors in
> the Makefile.  I don't think that there's any reason WHY it should require
> GNU Make, and if it is to be integrated into the lynx code at any point,
> this restriction should be removed if at all possible.  That's precisely
> why autoconfigure is so great is that it does not require the person
> building the software to have anything other than /bin/sh.
> 
> > I spent a lot of time getting the disk cache to work.  It does work
> > somewhat.  Well on the other hand last year I have argued (together with
> > Fote) that Lynx doesn't really need a disk cache (mostly because there are
> > external caching proxies that can be installed, which probably do a better
> > job of caching than Lynx could ever do.  because that is all they do.)
> > I still think Lynx doesn't really need one...
> 
> I rememeber this issue well.  I'm kinda stuck in the middle somewhere on
> this.  It would be nice if it came easily with some of the other things,
> but it sounds like it didn't. :)

If the networkin/protocol dependent layers of Lynx were structured
according to the new library's architecture, including the cache protocol
or not would be a relatively simple matter.  But (1) that is not the case
and (2) it would give only very basic functionality - it would require
lots of tweaks to deal with not-so-common situations, and to do caching
*right* in all those situations (one reason why RFC 2068 is so long is
caching...), then you need mechanisms to check for cache corruption, tools
to clean the cache, and make everything user configurable...

> > I think now that the most realistic approach for putting some of this back
> > into a public Lynx version is to start with some small (but extensive) 
> > changes - basically starting to redo the whole thing.  Like introducing
> > some new data structures (HTRequest, HTNet, HTResponse, ...) used
> > throughout the libwww5 code, changing all HTStream methods to return a
> > result everywhere in the Lynx code.  Those would be changes that would, 
> > at first, not change anything in what the code actually does.  But it
> > would then be easier to add other pieces; or grab parts from the existing
> > libwww5 and use them (maybe the DNS lookup cache...)
> 
> That's kind of what I was thinking of.  That was what was behind my
> question several months ago about the actual version number of the WWWLib
> in Lynx (says 2.14, but some of the stuff mentioned in the WWWLib's
> history file for 2.15 mention Lynx).  What I mentioned about the 8.3
> filename convention would be to go through the WWWLib's history file and
> note where they changed the naming system, and note which files they
> renamed and to what names and do the same for the lynx code.  I've got a
> list (on my linux box at home?) that I did like this.  There were still a
> couple of files with long names even after this.  I presume they were
> phased out before the name changes or lynx-specific.  Anyway, this would
> be one step, and what you mention would be another good one.  We should
> make this a goal.  Since you're a bit more intimate with the WWWLib 5
> code, perhaps you can make specific suggestions to those working on the
> lynx code as to what data structures, variables, etc. should be
> added/changed in the current code to pave the way for the new stuff.

Specific suggestions:

(1) Change all HTStream object methods to return a result int;
    add XXXX_flush() methods to all objects;
    make them pass result of downstream calls to caller.
    (following the way those things are defined in the news lib,
    see HTStream.h and HTStruct.h.

    That requires going through a lot of files (basically, grep for
    "HTStream_" and "HTStructured_", start changing those files,
    then try to compile and resolve errors - or similar).

I have those changes in my working messy code, but I would have to isolate
them from all the other changes I made, in order to put them into the
current 2.7...

(2) Start using a HTRequest * object to store information that is relevant
    for only one request, change some functions to use that (add to
    argument list, pass on to other functions, get and set info there
    instead of using global variables)

I'll stop here...

  Klaus

;
; To UNSUBSCRIBE:  Send a mail message to address@hidden
;                  with "unsubscribe lynx-dev" (without the
;                  quotation marks) on a line by itself.
;