[Duplicity-talk] changed files not being backed up

[Georg Lutz]

Hi,

I am experiencing that under certain conditions files are not being
backed up properly by an incremental run. This is of course a severe
problem, because these files cannot be restored until the next
fullbackup takes place.

Short story:

    $ mkdir data; mkdir backup
    $ echo a> data/a; echo b> data/b
    $ duplicity -v0 --no-print-statistics --no-encryption data 
file://backup
    $
    $ sleep 2; mv data/a data/b
    $ duplicity -v0 --no-print-statistics --no-encryption data 
file://backup
    $
    $ duplicity -v0 --no-print-statistics --no-encryption restore 
file://backup restore
    $ diff -r data restore
    diff -r data/b restore/b
    1c1
    < a
    ---
    > b
    $


Long story:

The problem occurs when the mtime for an already backed up file does not 
change. One example is a preserving copy action, another one are 
applications like digikam who reset mtime when they only change EXIF data.

Some month ago a similar problem has been reported:

 https://lists.gnu.org/archive/html/duplicity-talk/2011-06/msg00003.html

The 100% solution would be using checksums for comparison. Of course 
this is an expensive operation and not everyone can afford to wait 
forever for backups.

But strong checksumming is not necessary here. It would be enough to 
compare ctime instead of mtime.

Havent't tested it, but I suppose the change can be done quite easy in 
path.py, function __eq__ . What do you think?

This would fix the problem at least under posix like systems. To my 
knowdledge windows does not have anything like "inode change time" 
(ctime) but only "file creation time". I don't know how cygwin behaves, 
but maybe it sets the ctime also to the same value as mtime.

While ctime should be enough a more conservative approach could be to 
compare ctime, mtime and file size. Then we could catch at least the 
windows case where mtime has been preserved but file size differs.

But there is a strange comment in __eq__ regarding comparing signature
vs. file sizes. Shouldn't the file size be recorded in tar?


Regards
  Georg