[rdiff-backup-users] Hidden directories are ignored during incremental backup

[John covici]

I think this is a problem in the fact that * never includes anything
beginning wth . character.  To do this, you need to back up a superior
directory to the Mail directory and that will work -- I do it all the
time.

Hope this helps.

on Monday 06/01/2009 Cybertinus(address@hidden) wrote
 > Hello everybody,
 > 
 > This is my first post to this mailing list, so I hope I don't ask to many 
 > stupid 
 > questions ;). And also a short introduction:
 > I'm Cybertinus, a 22 year old IT-student from The Netherlands. I tend to 
 > fiddle 
 > around a bit with computers at home. Just playing a bit with Linux server 
 > aimed 
 > for server usage. Applications like Apache and MySQL, but also Nagios or 
 > Snort. 
 > Just to find out what it takes to get a secure, stable server up and 
 > running. 
 > And one of the most important parts of secure and stable is backups. And at 
 > my 
 > desktop at home I use rdiff-backup for the backups.
 > And when I create a new topic on a mailinglist or forum, I always seam to 
 > manage 
 > that is becomes a very long post :p, sry for that ;).
 > 
 > First let me explain what my backup plan is :) (you can skip this paragraph 
 > if 
 > you are only interested in the problem I have ;) ). I have 3 external 
 > harddrives. The first one is mounted on /backup and rdiff-backup writes the 
 > complete backup to that directory (well, /backup/snapshots actually).
 > The second disk is also connected to my computer, but then at the mountpoint 
 > /backup_sync. 15 minutes after rdiff-backup has updated the backup on 
 > /backup I 
 > use rsync to copy everything from /backup to /backup_sync (why? I tell you 
 > in a 
 > bit ;) )
 > The third disk is at my grandma's place, for external backup. Even when my 
 > house 
 > burns down, my data is safe :).
 > Once every week I disconnect the disk which was mounted at /backup and take 
 > that 
 > to my grandma. The disk that was there I take back home with me, and gets 
 > connected to /backup_sync. The disk that was at /backup_sync already moved 
 > to 
 > /backup, when the original disk at /backup was disconnected. This way I can 
 > rotate my disks and can I make incremental backups at my computer, and I 
 > have 
 > all the time in the world to swap the disks at my grandma's place :).
 > I've written a little script to automate the backup. I've created 3 
 > backupsets: 
 > the first set backups my most important directories every hour to my 
 > external 
 > disk, the first set backups the little bit less important directories every 
 > day 
 > and the rest of the data I want to keep gets backuped in the third set, once 
 > a week.
 > 
 > My problem I have with rdiff-backup is that it seams to ignore hidden files 
 > (files or directories that start with a .) when it creates a incremental 
 > backup. 
 > I use Courier IMAP as a mailserver and the mails are stored in a maildir. So 
 > I've got directories like ~/mail/.CybertinusNl.site1, 
 > ~/mail/.CybertinusNl.site2, etc. But those directories don't get incremented 
 > when I run rdiff-backup again. I've added a new maildir today 
 > (~/mail/.CybertinusNl.Backup) but that directory doesn't show up in the 
 > output 
 > directory of rdiff-backup.
 > 
 > I've written a script to make the configuration an generation of my backups 
 > very 
 > easy. This scripts is actually multiple files. The main file I called 
 > rdiff-main, and I've placed in /usr/sbin. It looks like this (don't mind the 
 > comments in the beginning, it is Dutch text, explaining the meaning of the 
 > script and stuff, nothing important):
 > #!/bin/bash
 > 
 > # Handige optie om incrementeel te backuppen...
 > # local of remote (default via ssh).
 > # Ik vind het handiger dan rsync of rsnapshot.
 > 
 > # Home: http://www.nongnu.org/rdiff-backup
 > # Voor Debian: in de repository
 > # Voor Red Hat Enterprise Linux/CentOS: bij Dag Wieers
 > # Voor Gentoo: in Porage
 > # Orgineel script: http://www.mrleejohn.nl/downloads/rdiff.txt
 > 
 > DEST="/backup/snapshots"
 > 
 > if [ -x /etc/rdiff-backup/rdiff-$1 ] ; then
 >      source /etc/rdiff-backup/rdiff-$1
 > else
 >      echo "The file rdiff-$1 doesn't exist. Please fix your config" 1>&2
 >      exit 1
 > fi
 > 
 > cd /
 > for dir in $DIRS ; do
 >      echo "Starting backup of $dir"
 >      rdiff-backup --create-full-path $dir "$DEST/$1/$dir"
 >      rdiff-backup --remove-older-than $TIME_SAVED "$DEST/$1/$dir"
 >      echo "Done backing up $dir"
 > done
 > 
 > As you can see, the script requires one argument, which defines which file 
 > needs 
 > to be included. That argument can be 'hourly' for example. 
 > /etc/rdiff-backup/rdiff-hourly looks like this:
 > #!/bin/bash
 > 
 > export DIRS='/home/mysql /home/svn /home/cybertinus/bash 
 > /home/cybertinus/localhost /home/cybertinus/mail'
 > export TIME_SAVED='2D'
 > 
 > So, just a configuration file, nothing fancy. But a neat way to create 
 > different 
 > backupsets ;).
 > When I first created the backup everything gets copied, but now I need it to 
 > increment the existing backup, but I don't see anything happening at 
 > /backup/snapshots/hourly/home/cybertinus/mail. The new directories don't get 
 > visible.
 > 
 > I've searched the manpage, the wiki and the mailinglist, but nothing told me 
 > anything about those hidden directories. I found one post on the mailinglist 
 > that told a user that you can use --exclude '**/.*' to skip all the hidden 
 > files. So I added --include '**/.*' to my main script, but then I got an 
 > error 
 > that rdiff-backup includes everything right away, so adding --include wasn't 
 > needed and that I probably meant something else, so the script just exited 
 > :(.
 > I also tried adding --backup-mode (with and without --force), but to no 
 > avail, 
 > the hidden directories still get ignored.
 > When I look at the directory with rdiff-backup -l 
 > /backup/snapshots/hourly/home/cybertinus/mail I see that it created a new 
 > snapshot. And normal files also got updated.
 > 
 > So, am I doing something wrong with rdiff-backup? Or don't I understand what 
 > I 
 > see in /backup/snapshots/hourly/home/cybertinus/mail? The last thing is very 
 > well possible, I just started using rdiff-backup (I used rsnapshot before 
 > this, 
 > but that wasn't exactly what I needed, and before that I didn't make any 
 > backups).
 > 
 > Thnx in advance for every reply :).
 > 
 > Best regards,
 > Cybertinus
 > 
 > 
 > _______________________________________________
 > rdiff-backup-users mailing list at address@hidden
 > http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
 > Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki

-- 
Your life is like a penny.  You're going to lose it.  The question is:
How do
you spend it?

         John Covici
         address@hidden