Re: lynx-dev Why doesn't lynx cache HTML source?

[Bela Lubkin]

David Combs wrote:

Before even starting, I would like to point out to David that I was
trying to provide an impartial summary of past arguments, not presenting
my own position.

> >   Con:
> > 
> >     - would add to the complexity of Lynx
> Huh?

Caching is complex, tricky business.  When this gets implemented we'll
surely see dozens of related bugs, for years afterwards.

> >     - caching rules are very difficult to get right
> 
> ie, "has the page changed since I last downloaded it".  At least for
> MY browsing, 99% of the time I don't care what's changed in the last
> TWO MINUTES or so.

In past discussions I have advocated a Lynx internal cache which would
hold many documents (user-settable), for arbitrary lengths of time --
perhaps even across invocations of Lynx.  As, for instance, Netscape
has.  Such a cache would have considerable benefits beyond the simple
1-document cache; but it would certainly require full, correct caching
semantics.

Even the simple 1-document cache has to interact with some caching
rules.  There have been bugs in the past relating to Lynx's current
1-document-in-1-rendering "cache", which is just about as simple as it
can possibly get.

> >     - would add code, increasing the size of Lynx source and binary
> 
> Come now.  EVERY improvement (almost) adds to size of source and binary,
> but you don't find too many of us complaining.  Most anyone using 
> LYNX instead of M$ or n-scrape are using for THREE reasons: SPEED, SPEED,
> and SPEED.  Many are administrators (system) and have SO MUCH WORK to 
> do in their 15-hour days, that they'll do most ANYTHING to save 5 seconds
> here, 3 seconds there -- when they do that operation 1000 times per day.

It is a valid objection which has been raised every time the subject has
been seriously discussed.

> >     - would increase the in-core and/or on-disk storage consumed by Lynx
> >       during operation
> 
> How much.  I mean, really, how much can it take to take input from
> a file than from the net?

Space to store that file on disk, or in memory.  Since there is no
natural limit on the size of an HTML source file, that could be
arbitrarily large.  (If the cache fails on files above a set size, I'm
sure you'll be back in a week complaining that you found a bigger
one...)

> >     - duplicates functionality which is already provided by other
> >       programs, i.e. web caches such as Squid -- programs which are
> >       dedicated to caching functionality and thus can be expected to do
> >       it better than Lynx could hope to
> Hey, I am using lynx as a USER, on an ISP over which I have NO control
> or even influence -- they won't even TALK to us users.

As LP said: get a better ISP.

> And if I had set up my own computer as a host, then you are suggesting
> that I go to all the trouble of finding, compiling, and (by FAR the 
> worst part of all), and having to LEARN some
> other program -- just so that LYNX (the ONLY program that I run
> that would involve such caching) can avoid downloading files.
> 
> Hey, it's hard enough to get LYNX to compile and work, and now
> you want me to to do that for some OTHER program!

Hey, you know what?  Lynx doesn't balance your checkbook, either, or
play `Go', or backup your system to tape.  Hate to break it to you, but
you will occasionally need software other than Lynx.

> >   Pro:
> > 
> >     - would add to the utility of Lynx
> >     - greatly speed operations which require a re-parse, including '\'
> >       view-source, '^V' other-DTD, '*' image-URLs, '"' soft-dquotes, '`'
> >       and "'" comment-parsing, '[' pseudo-alts, '@' raw-mode, and
> >       changes in assumed document character set.
> >     - easier for a regular user to install than a full web proxy
> 
> I'd say that "easier" is the wrong word.  It is virtually IMPOSSIBLE
> for a normal user to do that.  My own head is SO cluttered with 
> different stuff, that it is often in "thrashing mode".  Not myself
> being a "real" admin ('tho that is what I am required to be, as the
> only user on this sparcstation), you tell me how many classes would I
> have to go to, and pay for, and take the time and brain-space (scarce!)
> for, to learn to install AND MAINTAIN AND MANAGE such a thing.
> 
> NO THANKS!

Again, I was just passing on past arguments.  I agree with your
position: I could have a proxy, but haven't spent to time to build,
install, and learn one; complicated by the fact that I run Lynx on
dozens of systems which are in different physical locations, and some
have firewalls between them (where I have no control over firewall
behavior).  So a proxy would be very painful for me to use, and only
marginally useful.

I really want the internal Lynx cache.  So don't flame me for arguments
that aren't even mine.

>Bela<