bug-wget
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Bug-wget] wget fails to parse existing .listing files for FTP download


From: Mark
Subject: [Bug-wget] wget fails to parse existing .listing files for FTP download
Date: Tue, 2 Oct 2012 12:49:26 +0100
User-agent: SquirrelMail/1.4.21

Hi,

I'm using wget 1.13.4 and have come across a problem. Does this happen for
anyone else?

Situation:
 - Use wget to recursively mirror some directories on an FTP site. Use
--no-remove-listing so the .listing files generated for each directory
are preserved.
 - Part-way through the downloading process, break wget.
 - Later come back and resume mirroring using the same wget command.
 - wget fails to parse the already-downloaded .listing files and because
of that does not download any more files. (It actually tries to download
an empty/null filename.)

I'm using a command like this:
wget -x -S --no-clobber --recursive --level=inf --debug
--no-remove-listing --append-output=wget_log.txt --progress=dot:mega
"ftp://ftp.example.com/path/to/files/";

Would a workaround for the not-able-to-continue-mirroring problem be to
use --continue and not --no-clobber? Then wget should re-download the
.listing files, and because the already-downloaded files are all fully
downloaded, it should skip downloading them again.

Here's some example output.


DEBUG output created by Wget 1.13.4 on linux-gnu.

URI encoding = `UTF-8'
Using `ftp.microsoft.com/.listing' as listing tmp file.
File `ftp.microsoft.com/.listing' already there; not retrieving.
UNKNOWN; perms 0;
Skipping.
UNKNOWN; perms 0;
Skipping.
UNKNOWN; perms 0;
Skipping.
UNKNOWN; perms 0;
Skipping.
UNKNOWN; perms 0;
Skipping.
UNKNOWN; perms 0;
Skipping.
UNKNOWN; perms 0;
Skipping.
UNKNOWN; perms 0;
Skipping.
UNKNOWN; perms 0;
Skipping.
UNKNOWN; perms 0;
Skipping.
UNKNOWN; perms 0;
Skipping.
UNKNOWN; perms 0;
Skipping.
UNKNOWN; perms 0;
Skipping.
--2012-10-02 12:28:21--  ftp://ftp.microsoft.com/
           => `ftp.microsoft.com/index.html'
Resolving ftp.microsoft.com (ftp.microsoft.com)... 64.4.30.33
Caching ftp.microsoft.com => 64.4.30.33
Connecting to ftp.microsoft.com (ftp.microsoft.com)|64.4.30.33|:21...
connected.
Created socket 4.
Releasing 0x0000000001a9e430 (new refcount 1).
Logging in as anonymous ...
220 Microsoft FTP Service
--> USER anonymous

331 Anonymous access allowed, send identity (e-mail name) as password.
--> PASS Turtle Power!

230-Welcome to FTP.MICROSOFT.COM. Also visit
http://www.microsoft.com/downloads.
230 User logged in.
--> SYST

215 Windows_NT
--> PWD

257 "/" is current directory.
--> TYPE I

200 Type set to I.
==> CWD not needed.
--> SIZE

501 Invalid number of parameters.
conaddr is: 64.4.30.33
--> PASV

227 Entering Passive Mode (64,4,30,33,57,113).
trying to connect to 64.4.30.33 port 14705
Created socket 5.
--> RETR

501 Invalid number of parameters.

No such file `'.

Closed fd 5
Closed fd 4


The already-present .listing file in question is this (wget was able to
parse it the first time):
04-28-10  07:21PM       <DIR>          bussys
04-28-10  10:17PM       <DIR>          deskapps
04-28-10  11:14PM       <DIR>          developr
04-28-10  11:15PM       <DIR>          KBHelp
04-28-10  11:15PM       <DIR>          MISC
04-29-10  06:54AM       <DIR>          MISC1
04-29-10  08:18AM       <DIR>          mscomtest
04-29-10  08:47AM       <DIR>          peropsys
04-29-10  05:10PM       <DIR>          Products
04-29-10  05:13PM       <DIR>          PSS
04-29-10  05:22PM       <DIR>          ResKit
04-29-10  07:51PM       <DIR>          Services
04-30-10  08:37AM       <DIR>          Softlib


-- Mark





reply via email to

[Prev in Thread] Current Thread [Next in Thread]