Re: [Duplicity-talk] problem: duplicity is not backing up the my files

[Ed Blackman]

On Wed, Aug 13, 2014 at 04:58:09PM +0200, Jelle de Jong wrote:
> After some investigation I found that duplicity does not have the same
> files in its collection as the files in the location that should be
> backuped. I think it has to do that the files may be changed or
> removed after the duplicity command has been started. (since the
> backup takes a few days, and some files rotate every day).

I think duplicity backs up files as it comes to them, rather than 
scanning all files first, then backing them up.  If I'm correct, if a 
file changed or was deleted before duplicity came to it, duplicity 
wouldn't know that it was there when it started.

Backing up a live system is always risky because of just this problem: 
if backing up isn't instantaneous, your backup may not represent a 
consistent state of your system.  And a few days is a long way from 
instantaneous.

Some things you might want to consider:

The best option is to not back up your live system.  Use some sort of 
snapshot system (LVM or btrfs on Linux) to take an instantaneous 
snapshot, then back up the snapshot.

Failing that, you could change your duplicity backups to back up to a 
local disk, then sync that local disk to whatever network location 
you're currently backing up to.  I do that for my Amazon S3 backups, 
because like you a full backup for me would take several days to go 
directly to S3.

--
Ed Blackman