Re: [Duplicity-talk] Version 0.6.0 Released - Checkpoint/Restart

[Kenneth Loafman]

Michael Terry wrote:
> 2009/6/8 Kenneth Loafman <address@hidden>:
>> From now on, the --archive-dir option can be used to
>> change the location of the archive dir, but you are
>> responsible for moving the files if you change it.
> 
> I'm very excited about checkpoint/restore and I will be playing around
> with it shortly.  But I'm curious...  I recently talked on this list
> about suddenly turning on --archive-dir where past backups hadn't used
> an --archive-dir.
> 
> The result of that discussion was that it wouldn't work right now, but
> patches to make it work would be nice.
> 
> Does this new system run into that same problem?  i.e. if I have a
> backup that didn't have an archive-dir, will it now suddenly be forced
> to have an archive-dir and run into the cleanup/restore errors?
> 
> I haven't tested myself yet, but will later.

We're heading into an interesting discussion, but yes, you will need to
either do a full backup to initialize the archive-dir, or you will need
to download and decrypt your sig and manifest files in the last full
backup set.

The thing I realized before I finished this was that there needed to be
some way to 'name' a backup set.  Right now, you can use an option like:
--archive-dir=~/.duplicity/backup_name, but I'd like to move to the
point where each backup had its own directory, located by name, with all
the options stored in the directory along with the sig and manifest
files, but then I took it another step.

It would be handy to use a database for all of this, both the sig and
manifest info could be stored in the database, along with options, logs,
and history.  An SQL database could be used for this quite easily, but
my issue with database access is that now you're into major program
requirements that get away from small size.  Plus, to be a true backup,
you have to store the same info remotely so nothing is lost if the
backup machine dies, which kind of implies multiple formats.  I don't
think you'd want to shove a database over the wire for a backup.

Bottom line is that I'm strongly thinking of rewriting or at least doing
a major reorg on duplicity.  After going through all of this and getting
a fairly good understanding of duplicity, I think it has out-grown it's
original purpose and needs to either be pared down and simplified, or
enhanced and functionally completed into something that will handle
major backup tasks.

Some thoughts come to mind:
- Named backups, i.e. named directory or keyed database entries.
- Plugin style backends, i.e. we supply basic backends, others are able

  to be plugged in by just putting them in a plugin directory.
- Get away from tar for backend data structure.
- Allow incrementals to be folded forward into a new complete set.
- Differential backups, folded incrementals would be necessary for this.

Sorry, just rambling...

...Ken

signature.asc
Description: OpenPGP digital signature