Re: [rdiff-backup-users] Re:rdiff-backup for "hot" and "cold" backups of home directories, (Louis-Dominique Dubeau)

[Louis-Dominique Dubeau]

On Mon, 2008-06-02 at 11:45 +0800, Richard Chapman wrote: 
> Hi Louis
> 
> Have you considered using Logical Volume Management "Snapshot volumes" 
> to make you "hot" backups (arguably) totally reliable?

Thanks for the suggestion.

I use them for my current backups with rsync, which are all "cold".

I guess I could test LVM for hot backups but I think rdiff-backup's
default behavior may actually be desirable.  The idea would be that if a
file is being changed *during* a run of rdiff-backup, it is likely that
it will change again soon.  If file changes are clustered in time (and I
think they are), ideally I want to backup the file before a cluster of
changes and after a cluster of changes but not in the middle of a
cluster.

There's also the issue of an LVM snapshot filling up over its capacity
during a backup.  This is a real risk during a hot backup because hot
backups are supposed to happen behind the scenes without interrupting or
bugging the user.  It is a different matter for cold backups.  Since a
cold backup is intrusive to some extent (by design), the user gets a
reminder that they must be careful with changes to the filesystem as
long as the cold backup is running.  That's what I've done for a year
now and I haven't shot myself in the foot yet.  Of course, not everybody
is trustworthy and maybe I need to include the provision in my wrapper
script to force some users to wait until a cold backup is over before
doing anything else.  (Basically, the provision to not use an LVM
snapshot with these users.)

But here's a question, which applies to the use of LVM snapshots with
rdiff-backup in general.  Yesterday, I finally got to the point of
performing a full backup of my home with rdiff-backup.  It was a cold
backup so I created an LVM snapshot and backed up from that.  I hit a
bit a of snag when the time came to filter out the files I did not want
to back up.  With rsync, you can specify filters which are relative to
the source directory and anchor them to that directory.  For instance,
this rsync filter:

- /.Xauthority

excludes the file .Xauthority if it is inside the source directory.  My
source directory can be anything.  This comes in handy when backing up
from a snapshot.  If my normal source directory is

/home/ldd

and I create an LVM snapshot of the filesystem mounted at home and I
mount the snapshot at /media/backup, then I can specify a source of

/media/backup/ldd/

and rsync will work fine.  (Hmm... maybe I'm abusing the /media
directory by mounting an LVM snapshot there...) Basically, the same
filter works in both cases.  I have not been able to figure out a way to
achieve the same with rdiff-backup.  It could be that I don't understand
the inclusion/exclusion system of rdiff-backup or it could be that I've
encountered a real "problem".  (Whether it is a problem or a feature, I
don't know.)  This is not a major issue because I can include in my
wrapper script a way to adapt inclusions and exclusions depending on
whether I'm working from a snapshot or not but if there is a better way
to achieve source independence, I'd like to know.

It is possible to use ** but the exclusion becomes totally detached from
any position in the filesystem.  For instance:

- **/.Xauthority

Would exclude all .Xauthority files no matter where they appear in the
source.  For a file like .Xauthority, I'm not too worried but there may
be other kinds of files which I want to exclude from being backed up if
they appear in ~ but not if they appear somewhere else.

Thanks,
Louis