[Bug-wget] Timeout switch not working

[Ian Bradshaw]

Hi,

I am having some trouble with a wget task that is performing an HTTPS 
download of a large file (a 2.9GB 7Zip archive) every night. It is 
scheduled to run as part of an SSIS package, using the SSIS Execute 
Process task. The wget arguments passed are as follows (I've replaced 
sensitive information with the "x" character):

--http-user=SMP\xxxxxxxx --http-passwd=xxxxxxxx 
--output-document=C:\DB_Downloads\xxxxxxxx.7z 
--output-file=C:\DB_Downloads\log.txt --continue --timeout=300 
--tries=20 --no-check-certificate 
https://download.xxxxxxxx.net/xxxxxxxx/xxxxxxxx.7z

A typical scenario is for the SSIS package to start and the wget task 
starts logging information about the progress of the download. After 
some random interval (there is no discernible pattern in terms of time), 
the progress of the download will stop. For example, the download may 
start at 01:00 and stop at 01:07, as indicated by the modified date/time 
of the log file. Despite the timeout switch telling it to give up and 
try again after 5 minutes of no data received, this never happens. The 
wget task has been observed to run for hours (e.g. until after 9am) 
without progressing the download by a single byte and also not timing 
out or reporting any kind of error. The only way I can force a restart 
is to put a timeout setting on the SSIS Execute Process task and use SQL 
Server Agent to perform a TASKKILL on any active wget processes and 
restart the package. Invariably, when a new instance of the SSIS package 
starts, the download continues from where it left off and carries on, 
perhaps to completion, perhaps not.

I have run a number of ping tests over recent nights, which reveal that 
there is no significant loss of connectivity with the HTTPS server.

I am able to achieve a successful download of the file if I configure 
enough SQl Server Agent job steps to kill wget and launch another 
instance of the SSIS package. Sometimes the file will download 
successfully with one or two runs of the SSIS package, sometimes it 
needs up to five attempts. SSIS is currently configured to allow 90 
minutes for the download and the client site has a 10Mbps leased line to 
the Internet. There's no way to predict how much of the 90 minutes will 
be spent actively progressing the download and how many will be spent 
idle. For example, download attempt 1 may last for 30 minutes then 
become idle, download attempt 2 may last for the full 90 minutes, 
download attempt 3 may last for just 1 minute, then download attempt 4 
may successfully complete the download with further 20 minutes.

I am using the Win32 wget.exe from SourceForge and it's been working 
fine for the best part of two years. I think something has changed on 
the side of the download server but this is the domain of a third party. 
I've wondered if they have recently put in some kind of IPS that is 
randomly blocking the data packets but this is just speculation and the 
third party involved claim nothing has changed on their side. My 
question is: why does the timeout switch not help in this scenario and 
is this therefore a bug?

I have log files (with debug output) available, as well as the wgetrc 
file if this is useful. Any help or advice will be much appreciated.

Thanks,
Ian
--