Re: [Bug-wget] Multi-threaded getting of single file

[Tim Ruehsen]

On Wednesday 10 September 2014 18:26:45 Rajib Bandopadhyay wrote:
> Dear Developers,
> 
> Is it possible to have multi-threaded
> downloads in wget to accelerate
> downloading?

Wget is not multi-threaded, but being worked on (git branch 'parallel-wget').

> Is user-controlled multi-threading of a
> single file (the file may be large)
> download possible in wget?

You could start Wget several times in parallel together with the (new) '--
start-pos' option. Together with --quota, you possibly could download chunks 
of a file for later concatenation. Seems to be a hack and I did not test it.

Here is a snippet from the 'git log':
#####
commit dfa1f4e0646f6fab5b768fd67477601126070dbc
Author: Yousong Zhou <address@hidden>
Date:   Wed Mar 19 23:42:04 2014 +0800

    Make wget capable of starting downloads from a specified position.
    
    This patch adds an option `--start-pos' for specifying starting position
    of a HTTP or FTP download.
#####

If your server supports the 'metalink' format, missing or broken chunks could 
be automatically detected and downloaded - again basically implemented in 
parallel-wget.

> Is checking of the downloaded file
> possible with the server file using
> rsync like facility? If the file is
> corrupt is it possible to repair the
> damaged sections?

HTTP(S) nor FTP support this out-of-the-box.
You need a Metalink capable/configured server and client.
Metalink (http://www.metalinker.org/) seems not very widespread, though 
Fedora, Ubuntu and SuSE seem to support it for distribution downloads.

Tim