Re: [Bug-wget] unlink() before open()

[Micah Cowan]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Andre Majorel wrote:
> Hi Micah and folks. Thanks for taking over Wget.
> 
> Is there a way to make Wget delete a file before it clobbers it ?
> 
> That would be useful to make archived mirrors of a web site :
> 
>   mkdir 2009-01-01
>   wget -P 2009-01-01 -m http://fubar.com/
> 
> The next day :
> 
>   mkdir 2009-01-02
>   find 2009-01-01 | cpio -pdl 2009-01-02
>   wget -P 2009-01-02 -m --unlink http://fubar.com/
> 
> At that point, 2009-01-01/ and 2009-01-02/ are mirrors of what the
> site looked like on the same day. All files under 2009-01-02/ that
> haven't changed since the previous day are hard links to their
> counterpart under 2009-01-01/.

Hm. That's an interesting feature idea.

The main shortcoming I can think of is that this forces a race condition
between when we unlink the file and when we create the fresh one. Wget's
already kind of full of such race conditions, and it's generally frowned
upon to run wget in the same directory that other processes are writing
files in at the same time (say, another wget); but I'd like to move
Wget's code in more of a direction to avoid race conditions, rather than
introduce new ones.

Still, this would be understood as intended for a particular usage
scenario, and I can understand why it would be desirable.

Note, I'm liable to put a pretty low priority on this, but I doubt it'd
take much work at all, so if someone else wants to tackle this, be my
guest! :)

Tracking this at https://savannah.gnu.org/bugs/?24806

- --
Micah J. Cowan
Programmer, musician, typesetting enthusiast, gamer.
GNU Maintainer: wget, screen, teseq
http://micah.cowan.name/
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJGLfk7M8hyUobTrERAlrQAJ407WBPWgzoviG8r7YJuelx4qFEuQCfSDRc
Txu00C2NAnc55pVQd/+TiNU=
=NCyU
-----END PGP SIGNATURE-----