Re: [Bug-wget] mirroring from webpage: looking for --delete

[Micah Cowan]

On 12/16/2010 05:01 AM, Mojca Miklavec wrote:

<snip to main issue>

> however there is one problem ... whenever the server removes some
> files, they will be left in my folder. My main question is: how can I
> remove those files automatically (apart from rewriting wget in perl or
> ruby to suit my needs)?

<snip>

> Another option would be to loop over all the files on my local drive
> after wget finishes the job and check if they still exist on the
> remote server; and delete them if they don't. But that is all extra
> work that could probably easily be handled if I would find the right
> option in wget to delete those files instead of me.

I'd say this looping is probably your most viable option right now. To
avoid a round-trip for these "checks", I'd say use wget in --debug mode
and save the log somewhere. Then generate a list of all the paths you
have locally, and convert them into URLs. Then grep out from the wget
log all the instances of:

Deciding whether to enqueue
"http://micah.cowan.name/wp-content/themes/ocadia/images/bg.gif";.

(This is a debug message, so will appear in the above format, regardless
of the locale you've chosen to run wget in.)

And then check each of your local "URLs" to see if it appears somewhere
in one of those. AIUI, the URLs in the format I gave above are
guaranteed to be absolute URLs at this point (they've already been
resolved from potentially relative URLs).

This line makes a good one for fetching URLs from, because there is a
one-to-one correspondence between Wget parsing a valid URL, and issuing
this message (in debug mode).

> This list might not be the most appropriate place to ask this
> question, but I didn't know where else I could ask.

This list is the perfect place to ask. Unfortunately, I don't know how
helpful of an answer we can provide for this.

> Thank you very much,
>     Mojca

I have to wonder: is your name pronounced similarly to mine? ("My - kuh").

-- 
Micah J. Cowan
http://micah.cowan.name/