LYNX-DEV Re: responses - Warning, verbose

[Klaus Weide]

On Mon, 28 Apr 1997, Foteos Macrides wrote:
> Klaus Weide <address@hidden> wrote:
> >On Sun, 27 Apr 1997, Foteos Macrides wrote:
> >>[...]
> >>    [3] the logic of your recent, substantial additions to the
> >>        devel code for resolving fragments, for showing just the
> >>        fragment versus full URLs in the advanced statusline or
> >>        showinfo display, and for regulating re-use of cached
> >>        HText structures (which in my view is flawed to the extent
> >>        of representing "serious bugs" in the devel code, and
> >>        increases the memory requirements needlessly, to do things
> >>        that often shouldn't be done),
> >
> >Okay let's talk about this.  I was waiting for comments.  Also from
> >others who have tried the code, of course.
> >[...]
> 
>       When you add elements to the document, WWWDoc, and/or anchor
> structures, the allocations for them are retained in their stacks
> or hash tables, whereas if you can get away with just functions, those
> are loaded only whenever needed, without long-lasting allocations.  

I did not add anything to HTAnchor or TextAnchor structures, the type
field in the HTLink structure I use (for an HTAtom *) was basically
unused space wasted for a NULL pointer for every hyperlink.  I added
a BOOL to _document and nothing to WWWDoc; both these structures are
not used for multiple objects at any time, AFAICS.  The only structures
where I have added something and which may have many objects are
_histstruct (1 BOOL and 1 int) and _LYMapElement (1 BOOL).  The size
of these added structure members is small compared to the storage
required for other structure members, some of them individually
allocated strings for full URLs.

Comparing with Lynx/2.6, I see that since then _histstruct,
_document, and DocAddress have all got char * bookmark and BOOL safe
added to them.  The default MAXHIST got bumped up from 512 to 1024,
MAXLINKS from 256 to 1024 (instead of using a patch for dynamically
increasing links[]).  Lots of strings got added to HTParentAnchor
(among them several for date strings where a time_t could also do).
I also note that any move towards libwww5, should it happen, will
likely add a whole bunch of additions to existing structures, and
new kinds of objects kept in hash tables.  The chartrans code has
added new storage requirements.

I am not saying that conservative use of space isn't an important
goal, I am also not criticizing the additions (although I don't
understand some of them), but want to put this in some perspective.

> I
> had long ago added an LYoverride_no_cache flag, in conjunction with the
> handling I added for "Pragma: no-cache" or it's equivalent in HTTP/1.1
> Cache-Control directives via http/https headers and/or META tags.
> That acts in a complementary, coordinated way, with the
> LYforce_no_cache flag, in situations where it's unwieldy to use the
> latter, but do want to ignore no-cache directives, if they were
> present, as when acting on a PREV_DOC command, or a link activated
> via the History Page.  The vanilla code needed tweaks to use that
> properly in conjunction with use of fragments for positioning within
> the currently loaded document, so that they are equivalent to the
> keystroke comments for movement within the currently loaded document.
> The problem was not specific to replies from POSTs, though that is
> the most common situation which brings out the inadequacy in the
> vanilla code.  I had added a quicky fix in HTAccess.c, and then what
> I think is a comprehensive fix in fotemods, with minimal additional
> code and no additional long-lived allocations.
>
>       Fragments are solely instructions to clients, not part of the
> actual URL, and their presence or absence has no bearing on whether to
> honor no-cache directives when retrieving from cache (in our case, any
> HText structure which is not the one currently being displayed).  

So far we agree.  What we seem to disagree about is whether the presence
or absense of anything else *but* a fragment can change things.

> Also,
> whether the URL was resolved with or without anything preceding the
> fragment, and versus the current document's URL versus a different
> Content-Base header's, META tag's, or BASE tag's value, has no bearing
> on the matter once the HREF or SRC in the markup has been resolved.

This is a description of the current (2.7, 2.7.1, fotemods) handling.
How a URL reference is given cannot have any influence on further 
processing because there is no mechanism for it.  You give no reasons
why this should be so, or why it has to stay so.

>       Your fragment handling mods and INTERNAL_LINK_ANCHOR stuff
> go far beyond the issue of whether to back off from the Fielding
> URL draft when a MAP does not appear above the IMG or OBJECT which
> references it (see below), 

Right. (and I didn't say they had much to do with each other)

> create long-lived allocations needlessly,

See above; those extra bytes in the history list also are not central
to the whole thing, and maybe could be reduced or go away.

> and are based on unsound logic which contradicts the above.

You haven't shown that there is any unsound logic.  (If anybody
else sees it and just I don't, please let me know...)
As for contradiction: yes I disagree with your sentence above 
(starting with "Also,"), if that is meant to be taken as a rule rather
than a description.

>       For example try http://www.wfbr.edu/htbin/junk with the
> devel code versus the current lynx2-7-1+FOTEMODS.  That returns
> a "document" with a "Cache-Control: no-cache" header, a link
> to capsule.html in our public http server's data tree, and has
> two fragment-based links to the position of the capsule.html link
> within the same document.  One of the latter two has just
> HREF="#capsule" in the markup, the other has more than that, but
> they both *resolve* to exactly the same URL plus fragment directive.

I do not agree with your assumption that *resolving* an URL reference
into a full URL + fragment and then go from there, as Lynx traditionally
does, is all that is needed or all that is allowed.

>       lynx2-7-1+FOTEMODS treats them both as equivalent, and
> will not act on the no-cache directive if either is ACTIVATE-ed
> within that document (i.e., simply to reposition) but will if
> either is invoked when another document is being displayed, as
> it should, because you are retrieving from cache.
> 
>       With the devel code, the first, which had just "#capsule"
> as it's HREF value, will be displayed in the advance statusline and
> showinfo page as just the fragment, whereas the second, which had
> more in its HREF value, will be displayed as the actual URL plus
> fragment instruction, even though upon being resolved from the
> markup they are *functionally* identical.  If you ACTIVATE the
> first, you will be positioned at the capsule link without acting
> on the no-cache directive, as should happen, but if you ACTIVATE
> the second, the devel code will act on the no-cache directive,
> i.e., retrieve and render the document again before positioning
> you on the capsule link, as should NOT happen.

(... and, just for the record, which has happend for BOTH cases
since several versions back.)

Well, thanks for confirming that it is working as intended. :)

You claim that this "should NOT happen" but have given no sufficient basis
for this.  My basis for believing that this is what SHOULD happen, and
that Lynx was not so wrong, after all (for the second case):
- I looked at several RFCs/drafts/papers to find any rules on this.
  For example, from <URL: http://www.amazon.com/expires-report.html>:
    "If a resource is explicitly requested by clicking a link or submitting
    a form, it is <em>never</em> acceptable to display an expired copy[...]"
  I did not find anything that stated that a HREF is to be treated
  in any special way just because it has a URL (not just a fragment)
  and the URL coincides with the URL of the document where it appears.
  This includes current Internet Draft <URL:
  ftp://ftp.ietf.org/internet-drafts/draft-ietf-http-uahint-00.txt>
  which proposes new directives to control history list behavior. 
  Therefore I have to assume that the same rules apply as for an HREF
  with a URL that differs from that of the currently viewed document.
- The quoted sentence does not contradict the statements of the Fielding
  URL draft AND RFC 1866 (!) about the meaning of URL references which
  consist of a fragment ONLY: "clicking" such a link is NOT a request
  for a resource.
- I found at least one page that relies on Lynx 2.7 behavior, a browser
  test page (one that has been around for a while, and may well have
  influenced browser implementations):  try the test at <URL:
  http://www-1.openmarket.com/browsertest/cache/nph-linkpagecache.cgi>.
  Fotemods now fails that test, but 2.7 and current devel code pass.

>       Now try Scott's http://www.slcc.edu/index2.html with the
> devel code, and active the "[Info]" link to get and display the
> document Lynx creates internally for the USEMAP.  The MAP in
> index2.html has two AREA tags with just fragments as HREF values,
> so in the advanced statusline or showinfo page with the document Lynx
> creates for listing the MAP content as links (which is a *different*
> document from index2.html the way USEMAP handling is implemented
> in Lynx) they're shown as just "#new" and "#display", implying that
> if you ACTIVATE them you'll be repositioned within the currently
> displayed document, but in fact what will happen is that the
> HText stucture for index2.html will be retrieved from cache and
> you'll be positioned somewhere in that different document.

But, and I think I learned that from you, it is just an artifact of
the way things are handled by Lynx that the LYNXIMGMAP object appears
as a *different* document, a result of trying to deal as a text client
with something that is meant to be shown as an inlined part of another
page.  Moreover, there is no way to embed a link target in this
(artificial) currently displayed document.  So "#new" and "#display",
when seen in a LYNXIMGMAP, cannot possibly mean what you think they
imply.  Instead, they refer to internal link targets within the
document which contains the <MAP> which Lynx is trying to present, and
the address of that document is already visible at the top of the
LYNXIMGMAP screen.  Indeed, if you ACTIVATE such a link the HText
structure for that document will be retrieved from the cache if it is
there.  Now, this may not be the happiest of user interface choices,
and you may disagree with it, and it may change, or completely
disappear.  But please don't make the mistake of saying Lynx (devel)
has "actual bugs" just because you don't understand how a new feature
is supposed to work and draw wrong conclusions. :) :) :)

The "new feature" allows authors to make a distinction between the two
kinds of links you described further above (I am back to the normal,
non-MAP case, for simplicity), which both appear in your test page.
HREF="#capsule" means #capsule in the current document.
HREF="/htbin/junk#capsule" means #capsule in document /htbin/junk.
Normally this wouldn't make a difference when these HREFs appear in
document /htbin/junk, but it may make a difference if Lynx decides
that /htbin/junk should be non-cachable.  

Compare this to the processing for "Safe" headers which you added a
while ago, if you like.  Both add new structure members to the history
structure.  Both have some support by some obscure Internet Draft. 
Both are features not widely supported, but available to those  
who happen to know about them.

But, IMHO, this is not so much a new feature, it really is just doing
the right thing AND based on existing expectations of browser behavior.

>       I realize that it would be as much work to remove that
> INTERNAL_LINK_ANCHOR stuff as it was to generate and include in
> the devel code, but do consider that it really should go. :) :)

Well it is certainly possible to remove it, but you haven't given me
good reasons why it should go, and I haven't heard from others that it
should go.

The following is a different topic (so I have changed the order of
quoted paragraphs slightly):

>       Note also that for some reason you've removed my setting
> of the no_cache anchor element when creating the USEMAP list for
> display, thereby reviving the bug which Scott reported. [...]

I did not enable that change, because I was not sure whether this
wouldn't result in totally unnecessary new network requests in some
cases; so I left that for further study (which still hasn't ended...).
They ways the various caching-related global variables interact may be
"complementary" and "coordinated", but they are also a bit difficult
to understand.  Lynx already is sometimes abusive in its use of
"no-cache" on requests, using it in situations where it is not needed.
If you are unaware of that, I assume that you haven't been "surfing"
lately over a 14.4 line, with a proxy cache at the local side... I have. 
So I am a bit wary of adding things which look, to my imperfect
understanding, as if they could create more unnecessary no-cache
requests.  But I'll look into that, so be afraid... MORE BUGS ARE
COMING...

Now exactly HOW MANY smileys do I have to insert here to get the right
effect?  Still haven't learned that.

  Klaus

;
; To UNSUBSCRIBE:  Send a mail message to address@hidden
;                  with "unsubscribe lynx-dev" (without the
;                  quotation marks) on a line by itself.
;