Re: lynx-dev HTDoRead() HTTCP.c possible bug - retry limit set too high?

[Vlad Harchev]

On Fri, 7 Jul 2000, Klaus Weide wrote:

> On Fri, 7 Jul 2000, Vlad Harchev wrote:
> > On Fri, 7 Jul 2000, Klaus Weide wrote:
> > > On Fri, 7 Jul 2000, Vlad Harchev wrote:
> > > >   Seems we should add new lynx.cfg setting READ_TIMEOUT to control this 
> > > > (there
> > > > already exists CONNECT_TIMEOUT). Does anybody object against it? 
> > > 
> > > As long as you keep the current behavior by default...
> > 
> >   I assume that by "current behaviour" you mean current value of timeout.
> 
> I meant current behavior in the same situation.  What you call "current
> value of timeout" isn't all there is to it.

 I don't get you - I assume you understand that 'READ_TIMEOUT' addition will
just substitute '180000' with some expression - so all behaviour will remain
the same. 
  
> First of all, the 180000 wasn't origninally meant as a timeout.  Rather
> as a protection against an infinite loop, which is subtly different:

 Yes, this value defines the value of timeout (18000 seconds by default).
 
>     while (!ready) {
>         /*
>         **  Protect against an infinite loop.
>         */
>         if (tries++ >= 180000) {
>             HTAlert(gettext("Socket read failed for 180,000 tries."));
> 
> Secondly, note that not all systems will make use of your new READ_TIMEOUT
> anyway.  Only those for which the
> 
>     #define NETREAD  HTDoRead
> 
> is not overridden in www_tcp.h will.

  Yes, and it looks like cygwin and OS/2 will use HTDoRead.

> > > But this isn't really the best solution for the problem, if the problem
> > > is really: 'Non-interactive lynx processes hang around for too long
> > > under some conditions'.  The best solution, because it should always
> > > work, is: kill the process from the outside if it runs for too long.
> > > Shortening the read timeout, together with a short connect timeout,
> > > will still not apply to all situations.  I am thinking about some
> > > situations in the FTP protocol (if we are blocked in listen(), neither
> > > of the timeout applies), or reading a "local" file that is actually
> > > on an NFS-mounted filesystem that is unavailable, or some other special
> > > local files.
> > 
> >   As for NFS mounts - it's very counterintuitive to use CONNECT_TIMEOUT and
> > READ_TIMEOUT for reading "plain files" (as they seem to libc).
> 
> I didn't suggest to do that.

  OK.

> >  As for first part ("better use the script below") - we've discussed this
> > before. This won't work for crawling 
> 
> That is not the situation here.  The original poster mentioned -dump, no
> traversal.
> 
> Lynx's "traversal" code is quasi-interactive anyway.  You have a tty.
> A normal 'z' should work just fine to interrupt a hanging connect or
> read.  It did when I last checked.

  I didn't know about 'z'.
 
> > since lynx can't continue crawling from
> > the place it was interrupted. 
> 
> There are mechanisms to save enough state in the various files generated when
> traversing that traversal can be resumed.  At least that's what I thought,
> and what the message suggests that gets appended to one of the files when
> a traversing lynx is killed.

  I didn't know that.
 
> > Also, if we use the script, we can only limit
> > the total time of the crawling session, not the timeout for each individual
> > document.
> 
> True.
> 
> It depends on what the problem to be solved is (which nobody has clearly
> stated).  As I wrote explicitly, I assumed that

  Let's think about entire spectrum of problems with respect to timeout on
reading, not just described in original post.
 
> > > the problem
> > > is really: 'Non-interactive lynx processes hang around for too long
> > > under some conditions'.
> 
> If you are talking about "crawling session", you are talking about something
> else, apparently.  At least you're not talking about lynx with -dump.

  Yes, I was talking about recursively storing rendered versions of documents
recursively.

> >  Also the script won't work for MSDOS and will require 'kill', 'sh' for
> > other brain-damaged OSes or environments like Win* or OS/2 or Mac.
> 
> And will your READ_TIMEOUT work for all "brain-damaged OSes or
> environments like Win* or OS/2 or Mac"?  See NETREAD.

 It will be there for OS/2 and Cygwin apparently.

> > > Better learn how to kill a process so that it *never* can run longer
>     ====================================================================
> > > than a max time.  Take the shell script below as a starting point.
>     ===============
> 
> I stand by that.  Better learn how to do that, if that's what you need.
> 
> I didn't mean that a -read_timeout option would be useless.  Just that
> in the situation at hand, as well as others (but not all), it is not
> the most straightforward or reliable way to fullfil the requirement /
> solve the problem.

  Yes, that's what I mean - it won't me useless. But why do you think that it
will be not the most reliable way to lfil the requirement / solve the problem?

>   Klaus
> 
> 
> ; To UNSUBSCRIBE: Send "unsubscribe lynx-dev" to address@hidden
> 

 Best regards,
  -Vlad


; To UNSUBSCRIBE: Send "unsubscribe lynx-dev" to address@hidden