[Bug-wget] Batch retrieval does not recover from extended pause

[Pekka Pihlajasaari]

Hi,

I created a text file with about 1700 web pages to retrieve multiple PDF
files from a site. Some way through the batch retrieval I paused the
program (with Ctrl-S) on a Windows 7 64-bit machine expecting to
continue when I moved to another office.

About 10 hours later, I continued the process to receive the appended
responses. It seemed that the program recovered sufficiently from the
timeout to note that the file had appeared, but failed 19 retries to
retrieve it. The process then continued as expected with the remainder
of the list of URLs.

>From the behaviour this looks like the problem is related to recovery
from the extended pause that may interact with the ongoing connection
that was being held to the server.

The configuration has no .wgetrc file.

I tried to reproduce the effect by disconnecting the network connection
at the machine, but the application recovered without error. This seems
to be a boundary case where the session times out while the application
is stalled.

Regards,
Pekka Pihlajasaari
--
address@hidden  Data Abstraction (Pty) Ltd      +27 11 484 9664
--
100%[======================================>] 78,150      12.9K/s   in
5.3s

2012-03-26 10:22:30 (14.4 KB/s) - `J01602.pdf' saved [78150/78150]

--2012-03-26 10:22:30--
http://www.quartermaster.army.mil/jccoe/publications/re
cipes/section_J/J01700.pdf
Reusing existing connection to www.quartermaster.army.mil:80.
HTTP request sent, awaiting response... 200 OK
Length: 78053 (76K) [application/pdf]
Saving to: `J01700.pdf'

100%[======================================>] 78,053      13.9K/s   in
5.5s

2012-03-26 10:22:36 (13.9 KB/s) - `J01700.pdf' saved [78053/78053]

--2012-03-26 10:22:36--
http://www.quartermaster.army.mil/jccoe/publications/re
cipes/section_J/J01701.pdf
Reusing existing connection to www.quartermaster.army.mil:80.
HTTP request sent, awaiting response... 200 OK
Length: 78229 (76K) [application/pdf]
Saving to: `J01701.pdf'

 0% [                                       ] 0           --.-K/s   in
0s

2012-03-26 22:13:47 (0.00 B/s) - Read error at byte 0/78229 (Connection
reset by
 peer). Retrying.

--2012-03-26 22:13:48--  (try: 2)
http://www.quartermaster.army.mil/jccoe/publi
cations/recipes/section_J/J01701.pdf
Connecting to www.quartermaster.army.mil|143.84.161.3|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 78229 (76K) [application/pdf]
J01701.pdf has sprung into existence.
Retrying.

--2012-03-26 22:13:53--  (try: 3)
http://www.quartermaster.army.mil/jccoe/publi
cations/recipes/section_J/J01701.pdf
Connecting to www.quartermaster.army.mil|143.84.161.3|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 78229 (76K) [application/pdf]
J01701.pdf has sprung into existence.
Retrying.
--
--2012-03-26 22:16:40--  (try:20)
http://www.quartermaster.army.mil/jccoe/publi
cations/recipes/section_J/J01701.pdf
Connecting to www.quartermaster.army.mil|143.84.161.3|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 78229 (76K) [application/pdf]
J01701.pdf has sprung into existence.
Giving up.

--2012-03-26 22:16:41--
http://www.quartermaster.army.mil/jccoe/publications/re
cipes/section_J/J01800.pdf
Connecting to www.quartermaster.army.mil|143.84.161.3|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 79320 (77K) [application/pdf]
Saving to: `J01800.pdf'

100%[======================================>] 79,320      24.5K/s   in
3.2s

2012-03-26 22:16:45 (24.5 KB/s) - `J01800.pdf' saved [79320/79320]

--2012-03-26 22:16:45--
http://www.quartermaster.army.mil/jccoe/publications/re
cipes/section_J/J01801.pdf
Reusing existing connection to www.quartermaster.army.mil:80.
HTTP request sent, awaiting response... 200 OK
Length: 77357 (76K) [application/pdf]
Saving to: `J01801.pdf'

100%[======================================>] 77,357      2.86K/s   in
26s

2012-03-26 22:17:12 (2.86 KB/s) - `J01801.pdf' saved [77357/77357]

--2012-03-26 22:17:12--
http://www.quartermaster.army.mil/jccoe/publications/re