[Bug-wget] wget tries to download already fully-downloaded files (FTP)

[Mark]

Hi,

I noticed another issue related to my previous message "wget fails to
parse existing .listing files for FTP download"
http://lists.gnu.org/archive/html/bug-wget/2012-10/msg00000.html

When --continue is specified instead of --no-clobber, wget re-downloads
.listing files, which works around the issue I described in the above
message.

For already-downloaded files, their length matches that given in the
corresponding .listing file. Currently wget doesn't recognise that the
file is already fully downloaded, so it tries to download 0 bytes from the
file, sending REST [file length] / RETR [filename] commands to the FTP
server.

If resuming a previously interrupted mirroring session, all those useless
REST / RETR commands (one for each already-downloaded file) waste time and
put an unnecessary load on the FTP server.

wget should instead skip the file if the already-downloaded length matches
that in the .listing file.


Here's an example. The .listing file:
10-19-98  11:00PM              3680140 PSP1STRA.EXE
10-19-98  11:00PM              7344152 Psp1stra.exe.old
10-19-98  11:00PM              3680153 Psp1stra.exe.old2
10-19-98  11:00PM              3254911 PSP1STRI.EXE
10-19-98  11:00PM              3235080 Psp1stri.exe.old
10-19-98  11:00PM              3254924 Psp1stri.exe.old2
10-19-98  11:00PM                 6345 Readme.txt


Readme.txt was already downloaded. Here's the wget output for that file:

--2012-10-02 13:04:49-- 
ftp://ftp.microsoft.com/bussys/exchange/exchange-public/fixes/Eng/Exchg5.5/PostSP1/Store2-fix/Readme.txt
           =>
‘ftp.microsoft.com/bussys/exchange/exchange-public/fixes/Eng/Exchg5.5/PostSP1/Store2-fix/Readme.txt’
==> CWD not required.
conaddr is: 64.4.30.33
--> PASV

227 Entering Passive Mode (64,4,30,33,52,37).
trying to connect to 64.4.30.33 port 13349
Created socket 5.
--> REST 6345

350 Restarting at 6345.
--> RETR Readme.txt

125 Data connection already open; Transfer starting.
Length: 6345 (6.2K), 0 remaining

     0K
     0K                                                      100% 0.00 =0s

Closed fd 5
226 Transfer complete.


-- Mark