Re: lynx-dev HTDoRead() HTTCP.c possible bug - retry limit set too high?

[Klaus Weide]

On Fri, 7 Jul 2000, Vlad Harchev wrote:
> On Fri, 7 Jul 2000, Klaus Weide wrote:
> > On Fri, 7 Jul 2000, Vlad Harchev wrote:
> > >   Seems we should add new lynx.cfg setting READ_TIMEOUT to control this 
> > > (there
> > > already exists CONNECT_TIMEOUT). Does anybody object against it? 
> > 
> > As long as you keep the current behavior by default...
> 
>   I assume that by "current behaviour" you mean current value of timeout.

I meant current behavior in the same situation.  What you call "current
value of timeout" isn't all there is to it.

First of all, the 180000 wasn't origninally meant as a timeout.  Rather
as a protection against an infinite loop, which is subtly different:

    while (!ready) {
        /*
        **  Protect against an infinite loop.
        */
        if (tries++ >= 180000) {
            HTAlert(gettext("Socket read failed for 180,000 tries."));

Secondly, note that not all systems will make use of your new READ_TIMEOUT
anyway.  Only those for which the

    #define NETREAD  HTDoRead

is not overridden in www_tcp.h will.

> > But this isn't really the best solution for the problem, if the problem
> > is really: 'Non-interactive lynx processes hang around for too long
> > under some conditions'.  The best solution, because it should always
> > work, is: kill the process from the outside if it runs for too long.
> > Shortening the read timeout, together with a short connect timeout,
> > will still not apply to all situations.  I am thinking about some
> > situations in the FTP protocol (if we are blocked in listen(), neither
> > of the timeout applies), or reading a "local" file that is actually
> > on an NFS-mounted filesystem that is unavailable, or some other special
> > local files.
> 
>   As for NFS mounts - it's very counterintuitive to use CONNECT_TIMEOUT and
> READ_TIMEOUT for reading "plain files" (as they seem to libc).

I didn't suggest to do that.

>  As for first part ("better use the script below") - we've discussed this
> before. This won't work for crawling 

That is not the situation here.  The original poster mentioned -dump, no
traversal.

Lynx's "traversal" code is quasi-interactive anyway.  You have a tty.
A normal 'z' should work just fine to interrupt a hanging connect or
read.  It did when I last checked.

> since lynx can't continue crawling from
> the place it was interrupted. 

There are mechanisms to save enough state in the various files generated when
traversing that traversal can be resumed.  At least that's what I thought,
and what the message suggests that gets appended to one of the files when
a traversing lynx is killed.

> Also, if we use the script, we can only limit
> the total time of the crawling session, not the timeout for each individual
> document.

True.

It depends on what the problem to be solved is (which nobody has clearly
stated).  As I wrote explicitly, I assumed that

> > the problem
> > is really: 'Non-interactive lynx processes hang around for too long
> > under some conditions'.

If you are talking about "crawling session", you are talking about something
else, apparently.  At least you're not talking about lynx with -dump.

>  Also the script won't work for MSDOS and will require 'kill', 'sh' for
> other brain-damaged OSes or environments like Win* or OS/2 or Mac.

And will your READ_TIMEOUT work for all "brain-damaged OSes or
environments like Win* or OS/2 or Mac"?  See NETREAD.

> > Better learn how to kill a process so that it *never* can run longer
    ====================================================================
> > than a max time.  Take the shell script below as a starting point.
    ===============

I stand by that.  Better learn how to do that, if that's what you need.

I didn't mean that a -read_timeout option would be useless.  Just that
in the situation at hand, as well as others (but not all), it is not
the most straightforward or reliable way to fullfil the requirement /
solve the problem.

  Klaus


; To UNSUBSCRIBE: Send "unsubscribe lynx-dev" to address@hidden