Re: [Lynx-dev] Problems accessing wikipedia with lynx 2.8.8

[Thomas Dickey]

----- Original Message -----
| From: "Dick Wesseling" <address@hidden>
| To: address@hidden
| Sent: Thursday, October 26, 2017 9:14:01 PM
| Subject: Re: [Lynx-dev] Problems accessing wikipedia with lynx 2.8.8
| 
| 
| address@hidden wrote:
| 
| > About two weeks ago, lynx 2.8.8 rel 2, running on this Solaris
| > system,
| > ceased to be able to access wikipedia, although it had successfully
| > done
| > so for many years.
| >
| > Is this a known problem?  Any suggestions appreciated.
| 
| This is a known problem, known to me at least. Wikipedia used to
| trigger
| a bug in Lynx, then they apparently changed something which made Lynx
| work again, but recently it stopped working again.
| 
| The bug is that if a server sends a large amount of HTTP headers then
| Lynx does not read all of the data sent by the server and you get a
| truncated .gz file.


| 
| The reason is as follows:
| 
| - Network data is delivered to Lynx in chunks.
| 
| - HTLoadHTTP() reads the first chunk and processes the data in that
|   chunk, in particular the HTTP response line and the Content-Length
|   header.
|   At this moment the input stream can be positioned anywhere!
| 
| - Next HTLoadHTTP() calls HTCopy() to read and process the bulk
|   of the data. HTCopy() thinks it can limit the amount read to
|   anchor->content_length, but that is not true because the stream
|   is not positioned at the start of the content.
| 
|   However, HTCopy() usually gets away with this because it always
|   reads in portions of INPUT_BUFFER_SIZE. This mitigates the problem,
|   but not enough for Wikipedia.
| 
| The following patch solves the problem with Wikipedia, but it
| probably
| breaks other things.
| 
| --- lynx2.8.9dev.16/WWW/Library/Implementation/HTFormat.c.bak Sun Jul
|  2 19:09:45 2017
| +++ lynx2.8.9dev.16/WWW/Library/Implementation/HTFormat.c     Fri Oct 27
| 02:26:46 2017

sounds like this:

https://github.com/ThomasDickey/lynx-snapshots/commit/6d7cd77ddec64888f258f03f5b9efc9d57b09a0c#diff-1371112641c8cfcf23d0c819cb7959cc

http://lynx.invisible-island.net/current/CHANGES.html#v2.8.9dev.15

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=863008

-- 
Thomas E. Dickey <address@hidden>
http://invisible-island.net
ftp://invisible-island.net