[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Bug-wget] avoiding a large number of HEAD reqs when resuming
From: |
Giuseppe Scrivano |
Subject: |
Re: [Bug-wget] avoiding a large number of HEAD reqs when resuming |
Date: |
Fri, 01 May 2015 00:11:34 +0200 |
User-agent: |
Gnus/5.13 (Gnus v5.13) Emacs/24.4 (gnu/linux) |
address@hidden (User Goblin) writes:
> The situation: I'm trying to resume a large recursive download of a site
> with many files (-r -l 10 -c)
>
> The problem: When resuming, wget issues a large number of HEAD requests
> for each file that it already downloaded. This triggers the upstream firewall,
> making the download impossible.
have you had a look at --wait, --waitretry and --random-wait?
Maybe this is enough for circumventing your firewall, even though it
will slow down the download process.
Regards,
Giuseppe