Re: [Duplicity-talk] Incremental backup of files with changed data but unchanged timestamp

[Kenneth Loafman]

We don't need to do the librsync create delta, we just need to ignore the timestamp and duplicity will do that for us.

duplicity.py, dup_time.py, path.py, and tarfile.py are the ones that actually reference mtime.  path.py would be the place to look for comparison, well before we call librsync.  Take a look in path.py, line 325 for backup and line 372 for verify.  Replace with 'return 0' and you will alway go through the rdiff process.  Very expensive.

...Ken

On Sun, Aug 3, 2014 at 6:39 AM, <address@hidden> wrote:

hmm, some more searching didn't reveal no options for the librsync create delta call. it simply seems to create signatures for the whole file only.
 http://librsync.sourcefrog.net/doc/librsync.html#processing-whole-files

that suggests that the mtime is compared somewhere else, probably in
 http://bazaar.launchpad.net/~duplicity-team/duplicity/0.7-series/view/head:/duplicity/path.py
although i am absolutely clueless as to where in the code path this is supposed to happen.

@Ken, Mike: any (more) input?

after all this (identical mtime) comes up from time to time on the list e.g.
 https://lists.nongnu.org/archive/html/duplicity-talk/2013-07/msg00015.html
rsync allows to enforce checksum checking '-c' as well, so people probably will expect this from duplicity.

..ede

On 03.08.2014 13:05, Kenneth Loafman wrote:
> I've seen packages that have the timestamp reflect the version number, so
> he's probably right, it would be the packager doing the dirty trick.
>
> I'm fairly sure you are right that DeltaFile is the first place.  I could
> not find anything else.  Mod that and he should be good to go.  It will be
> a lot slower, so save the original for the next backup.
>
> ...Ken
>
>
>
> On Sun, Aug 3, 2014 at 5:11 AM, <address@hidden> wrote:
>
>> On 03.08.2014 02:03, Nate Eldredge wrote:
>>> I am using duplicity to make incremental backups of my system.  I have
>> some files whose data has changed since the last backup, but whose mtime
>> stayed the same.  It looks like `duplicity incremental' ignores files whose
>> timestamp has not changed, so it doesn't back up the new data.  Is there a
>> way to force duplicity to compare the file with a stored checksum, or even
>> to use rdiff unconditionally?  I'd prefer not to have to do a new full
>> backup.
>>>
>>> I'd consider hacking duplicity myself but it would be helpful to know
>> where in the code I should look.
>>>
>>> (Before you accuse me of abusing timestamps: it isn't my fault!  I
>> crossgraded this Ubuntu system from 32-bit to 64-bit.  It appears that some
>> Ubuntu packages have the same timestamps on corresponding files in the
>> 32-bit and 64-bit versions.  Presumably the packages were generated at the
>> same time, and coincidentally those files were compiled during the same
>> second.  So when I replaced the 32-bit package with the 64-bit package, I
>> get a different file with the same timestamp.)
>>>
>>> I'm using duplicity 0.6.23 (latest from the PPA) on Ubuntu 14.04.
>>>
>>
>> i like "(Before you accuse me of abusing timestamps: it isn't my fault!"
>> bit .. hehe as long as the time stamps were old enough you will get off
>> scott free i guess..
>>
>> but seriously - this was obviously not on the horizon of when duplicity
>> was developed. i searched a bit but couldn't find anything apart from the
>> librsync call 'librsync.DeltaFile(old_sigfp, newfp)' in
>>
>> http://bazaar.launchpad.net/~duplicity-team/duplicity/0.7-series/view/head:/duplicity/diffdir.py#L136
>>
>> i cannot seem to find a routine that checks time stamps before that.
>>
>> @Ken, Mike: can you hint where this magic happens?
>>
>> ..ede
>>
>