Re: [Bug-wget] Multi segment download

[Tim Rühsen]

Hi,

normally it makes much more sense when having several download mirrors and 
checksums for each chunk. The perfect technique for such is called 'Metalink' 
(more on www.metalinker.org).
Wget has it in branch 'master'. A GSOC project of Hubert Tarasiuk.

Additionally, Wget2 is under development, already having the option --chunk-
size (e.g. --chunk-size=1M) to start a multi-threaded download of a file.

Regards, Tim


Am Freitag, 28. August 2015, 15:41:27 schrieb Random Coder:
> On Fri, Aug 28, 2015 at 3:06 PM, Ander Juaristi <address@hidden> wrote:
> > Hi,
> > 
> > Would you point us to some potential use cases? How would a Wget user
> > benefit from such a feature? One of the best regarded feature of download
> > managers is the ability to resume paused downloads, and that's already
> > supported by Wget. Apart from that, I can't come across any other use
> > case. But that's me, maybe you have a broader overview.
> One possible feature, described in flowery language from a product
> description: "... splits files into several sections and downloads
> them simultaneously, allowing you to use any type of connection at the
> maximum available speed. With FDM download speed increases, or even
> more!"
> 
> And, just show this can help, at least in some situations, here's an
> example using curl (sorry, I don't know how to do a similar request in
> wget).  First a normal download of the file:
> 
> curl -o all http://mirror.internode.on.net/pub/test/100meg.test
> 
> This command takes an average of 48.9 seconds to run on my current
> network connection.  Now, if I split up the download as the download
> manager will, and run these four commands at the same instant:
> 
> curl -o part1 -r0-25000000
> http://mirror.internode.on.net/pub/test/100meg.test curl -o part2
> -r25000001-50000000
> http://mirror.internode.on.net/pub/test/100meg.test
> curl -o part3 -r50000001-75000000
> http://mirror.internode.on.net/pub/test/100meg.test
> curl -o part4 -r75000001-
> http://mirror.internode.on.net/pub/test/100meg.test
> 
> The union of time it takes all four commands to run ends up being an
> average of 19.9 seconds over a few test runs on the same connection.
> There's some penalty here because I need to spend time combining the
> files afterwards, but if the command supported this logic internally,
> no doubt much of that work could be done up front as the file is
> downloaded.

signature.asc
Description: This is a digitally signed message part.