Re: [Bug-wget] Timestamping vs incomplete downloads

[Tim Rühsen]

On 10/23/18 9:07 AM, Darshit Shah wrote:
> On October 22, 2018 11:49:12 PM UTC, Dave Warren <address@hidden> wrote:
>> Currently when a download with timestamping enabled gets interrupted, 
>> the timestamp of the resulting file ends up being the current time and 
>> when wget is re-executed after connectivity is restored the local file 
>> is then seen as newer and skipped.
>>
>> robocopy handles this a little differently, by setting a date far in
>> the 
>> past as a way of ensuring that on a subsequent execution the transfer 
>> can be resumed.
>>
>> Is there a better way to handle this situation in wget? A way to force 
>> an old date on the file? I'd be happy with a fixed "in the past" date, 
>> the service supplied date minus a second, etc. Or some way to detect 
>> that the file is incomplete (too small) on a subsequent run?
> 
> I haven't tested it but what you say indeed sounds like a valid bug.
> 
> The cleanest approach, IMO, is to use the extended file attributes in modern 
> systems to store this time at the very beginning and look for it on 
> continuation. Setting the time in the past doesn't work since every packet 
> that is written will once again update the last modified time. Setting the 
> time after each write() is not a feasible solution. What you suggest can only 
> work when the client gets a clean exit in the face of an interruption and 
> this isn't always the case. 

Good idea. Though he xattr stuff is not always available... but when it
is, we can indeed take advantage of it. I'll open an issue for wget2.

Regards, Tim

signature.asc
Description: OpenPGP digital signature