bug-wget
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Bug-wget] avoiding a large number of HEAD reqs when resuming


From: Giuseppe Scrivano
Subject: Re: [Bug-wget] avoiding a large number of HEAD reqs when resuming
Date: Fri, 01 May 2015 00:11:34 +0200
User-agent: Gnus/5.13 (Gnus v5.13) Emacs/24.4 (gnu/linux)

address@hidden (User Goblin) writes:

> The situation: I'm trying to resume a large recursive download of a site
> with many files (-r -l 10 -c)
>
> The problem: When resuming, wget issues a large number of HEAD requests
> for each file that it already downloaded. This triggers the upstream firewall,
> making the download impossible.

have you had a look at --wait, --waitretry and --random-wait?

Maybe this is enough for circumventing your firewall, even though it
will slow down the download process.

Regards,
Giuseppe



reply via email to

[Prev in Thread] Current Thread [Next in Thread]