[Duplicity-talk] Fixing a broken backup manually

[Milan Bouchet-Valat]

Hi!

I've got bitten by bug 579958 [1]: I ran out of free space while
performing an incremental backup. Then the clock error blocked
performing a backup again.

So I removed the .part files that were left, and the error changed: a
manifest file was not found. I don't remember whether it was the .gpg
or .gz version. I tried creating the needed manifest and signature files
manually, with gzip and gpg --encrypt, with no luck. I'm getting this
error when runnning incremental or cleanup:

$ duplicity cleanup file://.
GnuPG passphrase: 
Local and Remote metadata are synchronized, no sync needed.
Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1239, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1232, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1137, in main
    globals.archive_dir).set_values()
  File "/usr/lib/python2.6/dist-packages/duplicity/collections.py", line
681, in set_values
    self.get_backup_chains(partials + backend_filename_list)
  File "/usr/lib/python2.6/dist-packages/duplicity/collections.py", line
804, in get_backup_chains
    map(add_to_sets, filename_list)
  File "/usr/lib/python2.6/dist-packages/duplicity/collections.py", line
794, in add_to_sets
    if set.add_filename(filename):
  File "/usr/lib/python2.6/dist-packages/duplicity/collections.py", line
89, in add_filename
    self.set_manifest(filename)
  File "/usr/lib/python2.6/dist-packages/duplicity/collections.py", line
118, in set_manifest
    remote_filename)
AssertionError:
('duplicity-inc.20100428T100928Z.to.20100515T181051Z.manifest.gpg',
'duplicity-inc.20100428T100928Z.to.20100515T181051Z.manifest.gz')

If I move all the manifest.gz files (I have two) out of the dir, I get:

$ duplicity cleanup file://.
GnuPG passphrase: 
Local and Remote metadata are synchronized, no sync needed.
Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1239, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1232, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1137, in main
    globals.archive_dir).set_values()
  File "/usr/lib/python2.6/dist-packages/duplicity/collections.py", line
692, in set_values
    backup_chains)
  File "/usr/lib/python2.6/dist-packages/duplicity/collections.py", line
705, in set_matched_chain_pair
    sig_chains = sig_chains and self.get_sorted_chains(sig_chains)
  File "/usr/lib/python2.6/dist-packages/duplicity/collections.py", line
918, in get_sorted_chains
    assert len(chain_list) == 2
AssertionError


At this point, the directory contains:
$ ls
duplicity-full.20100428T100928Z.manifest.gpg
duplicity-full.20100428T100928Z.vol1.difftar.gz
duplicity-full.20100428T100928Z.vol2.difftar.gz
duplicity-full.20100428T100928Z.vol3.difftar.gz
duplicity-full.20100428T100928Z.vol4.difftar.gz
duplicity-full.20100428T100928Z.vol5.difftar.gz
duplicity-full.20100428T100928Z.vol6.difftar.gz
duplicity-full.20100428T100928Z.vol7.difftar.gz
duplicity-full.20100428T100928Z.vol8.difftar.gz
duplicity-full.20100428T100928Z.vol9.difftar.gz
duplicity-full-signatures.20100428T100928Z.sigtar.gpg
duplicity-full-signatures.20100428T100928Z.sigtar.gz
duplicity-inc.20100428T100928Z.to.20100515T181051Z.manifest.gpg
duplicity-inc.20100428T100928Z.to.20100515T181051Z.vol10.difftar.gpg
duplicity-inc.20100428T100928Z.to.20100515T181051Z.vol11.difftar.gpg
duplicity-inc.20100428T100928Z.to.20100515T181051Z.vol12.difftar.gpg
duplicity-inc.20100428T100928Z.to.20100515T181051Z.vol1.difftar.gpg
duplicity-inc.20100428T100928Z.to.20100515T181051Z.vol2.difftar.gpg
duplicity-inc.20100428T100928Z.to.20100515T181051Z.vol3.difftar.gpg
duplicity-inc.20100428T100928Z.to.20100515T181051Z.vol4.difftar.gpg
duplicity-inc.20100428T100928Z.to.20100515T181051Z.vol5.difftar.gpg
duplicity-inc.20100428T100928Z.to.20100515T181051Z.vol6.difftar.gpg
duplicity-inc.20100428T100928Z.to.20100515T181051Z.vol7.difftar.gpg
duplicity-inc.20100428T100928Z.to.20100515T181051Z.vol8.difftar.gpg
duplicity-inc.20100428T100928Z.to.20100515T181051Z.vol9.difftar.gpg

I know I may have made this a total mess, but TBH I don't really care
about the data in that backup. It would just be more practical for me to
be able to recover it, and BTW I'd like to make sure these assertion
errors are not bugs in duplicity.

So I have a few questions:
- Are the .manifest.gpg and .sigtar.gpg encrypted versions of the
original files? Should these files be removed from the dir?
- How is it possible that the full backup is not encrypted (no .gpg
extension) while incremental ones are?
- Finally, do you have ideas about how to fix my problem? Are there
missing files?

Sorry if I missed a documentation that would have explained this - if
it's not present, it would definitely be good to have, since people tend
to become nervous when something goes wrong about their backup. Knowing
how to fix it manually is always nice.

Anyway, nice program, I could have been smarter about what to do to fix
this, instead of messing around... (but why don't duplicity have an IRC
channel?). Thanks for your time!

Regards

1: https://bugs.launchpad.net/duplicity/+bug/579958