Re: [Duplicity-talk] Failures w/ S3 backend

[Maurizio Vitale]

Kenneth Loafman <address@hidden> writes:

Thank you very much for the pointers. It took some time to followup, but
here I am. For the sake of these experiments I've limited the systems
under consideration to two (I'll test others once problems have been
resolved here). They are:

         S - an openSuse 11.1 system where the backup is created
         T - an Ubuntu 8.10 system running in a vmware workstation VM
             [vmware doesn't seem to be art of the problem, as I was
              seeing the same on phisical machines and it is just more
              convenient to run experiments this way]

On S, I have:
   - duplicity 0.5.10
   - boto 1.6b

On T, I have:
   - duplicity 0.5.11 (also checked w/ 0.5.10, same result)
   - boto 1.6b

Experiments:

  I have made an incremental backup on S. Which means that on the Amazon
  server I have a chain of one full backup + one incremental backup. Did
  this to provide level 9 logs. All I see is the same as yesterday, so
  the additional inremental backup doesn't seem to be the problem

  I then tried to restore on S and T. 

  The same script has been used on
  both. On S I get a succesfull restore. The full log (@ verbosity 9)
  is around 30M. A trimmed one showing the beginning and ending is
  attached here

2009-03-11_11:20:00: Starting restore procedure for cuma (backup on aws). 
Restored to /home/mav/restored_backup/
2009-03-11_11:20:00: /usr/local/bin/duplicity restore --encrypt-key BD0CC45E 
--sign-key BD0CC45E --verbosity 9 s3+http://polymath/backup 
/home/mav/restored_backup/
Main action: restore
Listed 
s3+http://polymath/backup/duplicity-full-signatures.20090309T195921Z.sigtar.gpg
Listed s3+http://polymath/backup/duplicity-full.20090309T195921Z.manifest.gpg
Listed 
s3+http://polymath/backup/duplicity-full.20090309T195921Z.vol1.difftar.gpg
Listed 
s3+http://polymath/backup/duplicity-full.20090309T195921Z.vol10.difftar.gpg
Listed 
s3+http://polymath/backup/duplicity-full.20090309T195921Z.vol11.difftar.gpg
Listed 
s3+http://polymath/backup/duplicity-full.20090309T195921Z.vol12.difftar.gpg
Listed 
s3+http://polymath/backup/duplicity-full.20090309T195921Z.vol13.difftar.gpg
Listed 
s3+http://polymath/backup/duplicity-full.20090309T195921Z.vol14.difftar.gpg
Listed 
s3+http://polymath/backup/duplicity-full.20090309T195921Z.vol15.difftar.gpg
Listed 
s3+http://polymath/backup/duplicity-full.20090309T195921Z.vol16.difftar.gpg
Listed 
s3+http://polymath/backup/duplicity-full.20090309T195921Z.vol17.difftar.gpg
Listed 
s3+http://polymath/backup/duplicity-full.20090309T195921Z.vol18.difftar.gpg
Listed 
s3+http://polymath/backup/duplicity-full.20090309T195921Z.vol19.difftar.gpg
Listed 
s3+http://polymath/backup/duplicity-full.20090309T195921Z.vol2.difftar.gpg
Listed 
s3+http://polymath/backup/duplicity-full.20090309T195921Z.vol20.difftar.gpg
Listed 
s3+http://polymath/backup/duplicity-full.20090309T195921Z.vol21.difftar.gpg
Listed 
s3+http://polymath/backup/duplicity-full.20090309T195921Z.vol22.difftar.gpg
Listed 
s3+http://polymath/backup/duplicity-full.20090309T195921Z.vol23.difftar.gpg
Listed 
s3+http://polymath/backup/duplicity-full.20090309T195921Z.vol24.difftar.gpg
Listed 
s3+http://polymath/backup/duplicity-full.20090309T195921Z.vol25.difftar.gpg
Listed 
s3+http://polymath/backup/duplicity-full.20090309T195921Z.vol26.difftar.gpg
Listed 
s3+http://polymath/backup/duplicity-full.20090309T195921Z.vol27.difftar.gpg
Listed 
s3+http://polymath/backup/duplicity-full.20090309T195921Z.vol28.difftar.gpg
Listed 
s3+http://polymath/backup/duplicity-full.20090309T195921Z.vol29.difftar.gpg
Listed 
s3+http://polymath/backup/duplicity-full.20090309T195921Z.vol3.difftar.gpg
Listed 
s3+http://polymath/backup/duplicity-full.20090309T195921Z.vol30.difftar.gpg
Listed 
s3+http://polymath/backup/duplicity-full.20090309T195921Z.vol31.difftar.gpg
Listed 
s3+http://polymath/backup/duplicity-full.20090309T195921Z.vol32.difftar.gpg
Listed 
s3+http://polymath/backup/duplicity-full.20090309T195921Z.vol33.difftar.gpg
Listed 
s3+http://polymath/backup/duplicity-full.20090309T195921Z.vol34.difftar.gpg
Listed 
s3+http://polymath/backup/duplicity-full.20090309T195921Z.vol35.difftar.gpg
Listed 
s3+http://polymath/backup/duplicity-full.20090309T195921Z.vol36.difftar.gpg
Listed 
s3+http://polymath/backup/duplicity-full.20090309T195921Z.vol37.difftar.gpg
Listed 
s3+http://polymath/backup/duplicity-full.20090309T195921Z.vol38.difftar.gpg
Listed 
s3+http://polymath/backup/duplicity-full.20090309T195921Z.vol39.difftar.gpg
Listed 
s3+http://polymath/backup/duplicity-full.20090309T195921Z.vol4.difftar.gpg
Listed 
s3+http://polymath/backup/duplicity-full.20090309T195921Z.vol40.difftar.gpg
Listed 
s3+http://polymath/backup/duplicity-full.20090309T195921Z.vol41.difftar.gpg
Listed 
s3+http://polymath/backup/duplicity-full.20090309T195921Z.vol42.difftar.gpg
Listed 
s3+http://polymath/backup/duplicity-full.20090309T195921Z.vol43.difftar.gpg
Listed 
s3+http://polymath/backup/duplicity-full.20090309T195921Z.vol44.difftar.gpg
Listed 
s3+http://polymath/backup/duplicity-full.20090309T195921Z.vol45.difftar.gpg
Listed 
s3+http://polymath/backup/duplicity-full.20090309T195921Z.vol46.difftar.gpg
Listed 
s3+http://polymath/backup/duplicity-full.20090309T195921Z.vol47.difftar.gpg
Listed 
s3+http://polymath/backup/duplicity-full.20090309T195921Z.vol5.difftar.gpg
Listed 
s3+http://polymath/backup/duplicity-full.20090309T195921Z.vol6.difftar.gpg
Listed 
s3+http://polymath/backup/duplicity-full.20090309T195921Z.vol7.difftar.gpg
Listed 
s3+http://polymath/backup/duplicity-full.20090309T195921Z.vol8.difftar.gpg
Listed 
s3+http://polymath/backup/duplicity-full.20090309T195921Z.vol9.difftar.gpg
Listed 
s3+http://polymath/backup/duplicity-inc.20090309T195921Z.to.20090311T122843Z.manifest.gpg
Listed 
s3+http://polymath/backup/duplicity-inc.20090309T195921Z.to.20090311T122843Z.vol1.difftar.gpg
Listed 
s3+http://polymath/backup/duplicity-new-signatures.20090309T195921Z.to.20090311T122843Z.sigtar.gpg
52 files exist on backend
Extracting backup chains from list of files: 
['duplicity-full-signatures.20090309T195921Z.sigtar.gpg', 
'duplicity-full.20090309T195921Z.manifest.gpg', 
'duplicity-full.20090309T195921Z.vol1.difftar.gpg', 
'duplicity-full.20090309T195921Z.vol10.difftar.gpg', 
'duplicity-full.20090309T195921Z.vol11.difftar.gpg', 
'duplicity-full.20090309T195921Z.vol12.difftar.gpg', 
'duplicity-full.20090309T195921Z.vol13.difftar.gpg', 
'duplicity-full.20090309T195921Z.vol14.difftar.gpg', 
'duplicity-full.20090309T195921Z.vol15.difftar.gpg', 
'duplicity-full.20090309T195921Z.vol16.difftar.gpg', 
'duplicity-full.20090309T195921Z.vol17.difftar.gpg', 
'duplicity-full.20090309T195921Z.vol18.difftar.gpg', 
'duplicity-full.20090309T195921Z.vol19.difftar.gpg', 
'duplicity-full.20090309T195921Z.vol2.difftar.gpg', 
'duplicity-full.20090309T195921Z.vol20.difftar.gpg', 
'duplicity-full.20090309T195921Z.vol21.difftar.gpg', 
'duplicity-full.20090309T195921Z.vol22.difftar.gpg', 
'duplicity-full.20090309T195921Z.vol23.difftar.gpg', 
'duplicity-full.20090309T195921Z.vol24.difftar.gpg', 
'duplicity-full.20090309T195921Z.vol25.difftar.gpg', 
'duplicity-full.20090309T195921Z.vol26.difftar.gpg', 
'duplicity-full.20090309T195921Z.vol27.difftar.gpg', 
'duplicity-full.20090309T195921Z.vol28.difftar.gpg', 
'duplicity-full.20090309T195921Z.vol29.difftar.gpg', 
'duplicity-full.20090309T195921Z.vol3.difftar.gpg', 
'duplicity-full.20090309T195921Z.vol30.difftar.gpg', 
'duplicity-full.20090309T195921Z.vol31.difftar.gpg', 
'duplicity-full.20090309T195921Z.vol32.difftar.gpg', 
'duplicity-full.20090309T195921Z.vol33.difftar.gpg', 
'duplicity-full.20090309T195921Z.vol34.difftar.gpg', 
'duplicity-full.20090309T195921Z.vol35.difftar.gpg', 
'duplicity-full.20090309T195921Z.vol36.difftar.gpg', 
'duplicity-full.20090309T195921Z.vol37.difftar.gpg', 
'duplicity-full.20090309T195921Z.vol38.difftar.gpg', 
'duplicity-full.20090309T195921Z.vol39.difftar.gpg', 
'duplicity-full.20090309T195921Z.vol4.difftar.gpg', 
'duplicity-full.20090309T195921Z.vol40.difftar.gpg', 
'duplicity-full.20090309T195921Z.vol41.difftar.gpg', 
'duplicity-full.20090309T195921Z.vol42.difftar.gpg', 
'duplicity-full.20090309T195921Z.vol43.difftar.gpg', 
'duplicity-full.20090309T195921Z.vol44.difftar.gpg', 
'duplicity-full.20090309T195921Z.vol45.difftar.gpg', 
'duplicity-full.20090309T195921Z.vol46.difftar.gpg', 
'duplicity-full.20090309T195921Z.vol47.difftar.gpg', 
'duplicity-full.20090309T195921Z.vol5.difftar.gpg', 
'duplicity-full.20090309T195921Z.vol6.difftar.gpg', 
'duplicity-full.20090309T195921Z.vol7.difftar.gpg', 
'duplicity-full.20090309T195921Z.vol8.difftar.gpg', 
'duplicity-full.20090309T195921Z.vol9.difftar.gpg', 
'duplicity-inc.20090309T195921Z.to.20090311T122843Z.manifest.gpg', 
'duplicity-inc.20090309T195921Z.to.20090311T122843Z.vol1.difftar.gpg', 
'duplicity-new-signatures.20090309T195921Z.to.20090311T122843Z.sigtar.gpg']
File duplicity-full-signatures.20090309T195921Z.sigtar.gpg is not part of a 
known set; creating new set
Ignoring file (rejected by backup set) 
'duplicity-full-signatures.20090309T195921Z.sigtar.gpg'
File duplicity-full.20090309T195921Z.manifest.gpg is not part of a known set; 
creating new set
File duplicity-full.20090309T195921Z.vol1.difftar.gpg is part of known set
File duplicity-full.20090309T195921Z.vol10.difftar.gpg is part of known set
File duplicity-full.20090309T195921Z.vol11.difftar.gpg is part of known set
File duplicity-full.20090309T195921Z.vol12.difftar.gpg is part of known set
File duplicity-full.20090309T195921Z.vol13.difftar.gpg is part of known set
File duplicity-full.20090309T195921Z.vol14.difftar.gpg is part of known set
File duplicity-full.20090309T195921Z.vol15.difftar.gpg is part of known set
File duplicity-full.20090309T195921Z.vol16.difftar.gpg is part of known set
File duplicity-full.20090309T195921Z.vol17.difftar.gpg is part of known set
File duplicity-full.20090309T195921Z.vol18.difftar.gpg is part of known set
File duplicity-full.20090309T195921Z.vol19.difftar.gpg is part of known set
File duplicity-full.20090309T195921Z.vol2.difftar.gpg is part of known set
File duplicity-full.20090309T195921Z.vol20.difftar.gpg is part of known set
File duplicity-full.20090309T195921Z.vol21.difftar.gpg is part of known set
File duplicity-full.20090309T195921Z.vol22.difftar.gpg is part of known set
File duplicity-full.20090309T195921Z.vol23.difftar.gpg is part of known set
File duplicity-full.20090309T195921Z.vol24.difftar.gpg is part of known set
File duplicity-full.20090309T195921Z.vol25.difftar.gpg is part of known set
File duplicity-full.20090309T195921Z.vol26.difftar.gpg is part of known set
File duplicity-full.20090309T195921Z.vol27.difftar.gpg is part of known set
File duplicity-full.20090309T195921Z.vol28.difftar.gpg is part of known set
File duplicity-full.20090309T195921Z.vol29.difftar.gpg is part of known set
File duplicity-full.20090309T195921Z.vol3.difftar.gpg is part of known set
File duplicity-full.20090309T195921Z.vol30.difftar.gpg is part of known set
File duplicity-full.20090309T195921Z.vol31.difftar.gpg is part of known set
File duplicity-full.20090309T195921Z.vol32.difftar.gpg is part of known set
File duplicity-full.20090309T195921Z.vol33.difftar.gpg is part of known set
File duplicity-full.20090309T195921Z.vol34.difftar.gpg is part of known set
File duplicity-full.20090309T195921Z.vol35.difftar.gpg is part of known set
File duplicity-full.20090309T195921Z.vol36.difftar.gpg is part of known set
File duplicity-full.20090309T195921Z.vol37.difftar.gpg is part of known set
File duplicity-full.20090309T195921Z.vol38.difftar.gpg is part of known set
File duplicity-full.20090309T195921Z.vol39.difftar.gpg is part of known set
File duplicity-full.20090309T195921Z.vol4.difftar.gpg is part of known set
File duplicity-full.20090309T195921Z.vol40.difftar.gpg is part of known set
File duplicity-full.20090309T195921Z.vol41.difftar.gpg is part of known set
File duplicity-full.20090309T195921Z.vol42.difftar.gpg is part of known set
File duplicity-full.20090309T195921Z.vol43.difftar.gpg is part of known set
File duplicity-full.20090309T195921Z.vol44.difftar.gpg is part of known set
File duplicity-full.20090309T195921Z.vol45.difftar.gpg is part of known set
File duplicity-full.20090309T195921Z.vol46.difftar.gpg is part of known set
File duplicity-full.20090309T195921Z.vol47.difftar.gpg is part of known set
File duplicity-full.20090309T195921Z.vol5.difftar.gpg is part of known set
File duplicity-full.20090309T195921Z.vol6.difftar.gpg is part of known set
File duplicity-full.20090309T195921Z.vol7.difftar.gpg is part of known set
File duplicity-full.20090309T195921Z.vol8.difftar.gpg is part of known set
File duplicity-full.20090309T195921Z.vol9.difftar.gpg is part of known set
File duplicity-inc.20090309T195921Z.to.20090311T122843Z.manifest.gpg is not 
part of a known set; creating new set
File duplicity-inc.20090309T195921Z.to.20090311T122843Z.vol1.difftar.gpg is 
part of known set
File duplicity-new-signatures.20090309T195921Z.to.20090311T122843Z.sigtar.gpg 
is not part of a known set; creating new set
Ignoring file (rejected by backup set) 
'duplicity-new-signatures.20090309T195921Z.to.20090311T122843Z.sigtar.gpg'
Found backup chain [Mon Mar  9 15:59:21 2009]-[Mon Mar  9 15:59:21 2009]
Added incremental Backupset (start_time: Mon Mar  9 15:59:21 2009 / end_time: 
Wed Mar 11 08:28:43 2009)
Added set Wed Mar 11 08:28:43 2009 to pre-existing chain [Mon Mar  9 15:59:21 
2009]-[Wed Mar 11 08:28:43 2009]
Collection Status
-----------------
Connecting with backend: BotoBackend
Archive dir: None

Found 0 backup chains without signatures.

Found a complete backup chain with matching signature chain:
-------------------------
Chain start time: Mon Mar  9 15:59:21 2009
Chain end time: Wed Mar 11 08:28:43 2009
Number of contained backup sets: 2
Total number of contained volumes: 48
 Type of backup set:                            Time:      Num volumes:
                Full         Mon Mar  9 15:59:21 2009                47
         Incremental         Wed Mar 11 08:28:43 2009                 1
-------------------------
No orphaned or incomplete backup sets found.
Last full backup date: Mon Mar  9 15:59:21 2009
Using temporary directory /tmp/duplicity-oOXh-p-tempdir
Registering (mktemp) temporary file 
/tmp/duplicity-oOXh-p-tempdir/mktemp-tTKH5G-1
Downloading 
s3+http://polymath/backup/duplicity-full.20090309T195921Z.manifest.gpg
gpg: encrypted with 2048-bit ELG key, ID B55457F8, created 2009-01-12
      "Maurizio Vitale (Key for Polymath Solutions off-site backups) 
<address@hidden>"
gpg: Signature made Mon 09 Mar 2009 08:01:50 PM EDT using DSA key ID BD0CC45E
gpg: Good signature from "Maurizio Vitale (Key for Polymath Solutions off-site 
backups) <address@hidden>"
Deleting /tmp/duplicity-oOXh-p-tempdir/mktemp-tTKH5G-1
Forgetting temporary file /tmp/duplicity-oOXh-p-tempdir/mktemp-tTKH5G-1
Registering (mktemp) temporary file 
/tmp/duplicity-oOXh-p-tempdir/mktemp-95fUDS-2
Downloading 
s3+http://polymath/backup/duplicity-full.20090309T195921Z.vol1.difftar.gpg
Registering (mktemp) temporary file 
/tmp/duplicity-oOXh-p-tempdir/mktemp-FG7seS-3
Downloading 
s3+http://polymath/backup/duplicity-inc.20090309T195921Z.to.20090311T122843Z.manifest.gpg
gpg: encrypted with 2048-bit ELG key, ID B55457F8, created 2009-01-12
      "Maurizio Vitale (Key for Polymath Solutions off-site backups) 
<address@hidden>"
gpg: Signature made Wed 11 Mar 2009 08:34:31 AM EDT using DSA key ID BD0CC45E
gpg: Good signature from "Maurizio Vitale (Key for Polymath Solutions off-site 
backups) <address@hidden>"
Deleting /tmp/duplicity-oOXh-p-tempdir/mktemp-FG7seS-3
Forgetting temporary file /tmp/duplicity-oOXh-p-tempdir/mktemp-FG7seS-3
Registering (mktemp) temporary file 
/tmp/duplicity-oOXh-p-tempdir/mktemp-8U3K7I-4
Downloading 
s3+http://polymath/backup/duplicity-inc.20090309T195921Z.to.20090311T122843Z.vol1.difftar.gpg
Making directory /home/mav/restored_backup
Writing .metadata of type dir
Making directory /home/mav/restored_backup/.metadata
Writing .metadata/.lock of type reg
Writing .metadata/.log of type reg
Writing .metadata/.plugins of type dir

... lot of stuff deleted ...

gpg: encrypted with 2048-bit ELG key, ID B55457F8, created 2009-01-12
      "Maurizio Vitale (Key for Polymath Solutions off-site backups) 
<address@hidden>"
gpg: Signature made Mon 09 Mar 2009 07:58:04 PM EDT using DSA key ID BD0CC45E
gpg: Good signature from "Maurizio Vitale (Key for Polymath Solutions off-site 
backups) <address@hidden>"
Deleting /tmp/duplicity-oOXh-p-tempdir/mktemp-6Y4p-h-50
Forgetting temporary file /tmp/duplicity-oOXh-p-tempdir/mktemp-6Y4p-h-50
Processed volume 48 of 48
2009-03-11_17:56:38: Starting restore procedure for cuma (backup on aws). 
Restored to /home/mav/restored_backup/
2009-03-11_17:56:38: /usr/local/bin/duplicity restore --encrypt-key BD0CC45E 
--sign-key BD0CC45E --verbosity 9 file://home/mav/copy_of_remote_desktop 
/home/mav/restored_backup/
Main action: restore
0 files exist on backend
Extracting backup chains from list of files: []
Collection Status
-----------------
Connecting with backend: LocalBackend
Archive dir: None

Found 0 backup chains without signatures.
No backup chains with active signatures found
No orphaned or incomplete backup sets found.
Last full backup date: none
Using temporary directory /tmp/duplicity-a6FFct-tempdir
Traceback (most recent call last):
  File "/usr/local/bin/duplicity", line 589, in <module>
    with_tempdir(main)
  File "/usr/local/bin/duplicity", line 582, in with_tempdir
    fn()
  File "/usr/local/bin/duplicity", line 544, in main
    restore(col_stats)
  File "/usr/local/bin/duplicity", line 304, in restore
    restore_get_patched_rop_iter(col_stats)):
  File "/usr/local/bin/duplicity", line 321, in restore_get_patched_rop_iter
    backup_chain = col_stats.get_backup_chain_at_time(time)
  File "/usr/local/lib64/python2.5/site-packages/duplicity/collections.py", 
line 798, in get_backup_chain_at_time
    raise CollectionsError("No backup chains found")
CollectionsError: No backup chains found

2009-03-11_17:58:53: Starting restore procedure for cuma (backup on aws). 
Restored to /home/mav/restored_backup/
2009-03-11_17:58:53: /usr/local/bin/duplicity restore --encrypt-key BD0CC45E 
--sign-key BD0CC45E --verbosity 9 file://home/mav/copy_of_remote_backup 
/home/mav/restored_backup/
Main action: restore
0 files exist on backend
Extracting backup chains from list of files: []
Collection Status
-----------------
Connecting with backend: LocalBackend
Archive dir: None

Found 0 backup chains without signatures.
No backup chains with active signatures found
No orphaned or incomplete backup sets found.
Last full backup date: none
Using temporary directory /tmp/duplicity-GIJBBi-tempdir
Traceback (most recent call last):
  File "/usr/local/bin/duplicity", line 589, in <module>
    with_tempdir(main)
  File "/usr/local/bin/duplicity", line 582, in with_tempdir
    fn()
  File "/usr/local/bin/duplicity", line 544, in main
    restore(col_stats)
  File "/usr/local/bin/duplicity", line 304, in restore
    restore_get_patched_rop_iter(col_stats)):
  File "/usr/local/bin/duplicity", line 321, in restore_get_patched_rop_iter
    backup_chain = col_stats.get_backup_chain_at_time(time)
  File "/usr/local/lib64/python2.5/site-packages/duplicity/collections.py", 
line 798, in get_backup_chain_at_time
    raise CollectionsError("No backup chains found")
CollectionsError: No backup chains found

  The duplicity command run (also logged) is:
  /usr/local/bin/duplicity restore --encrypt-key BD0CC45E --sign-key BD0CC45E 
--verbosity 9 s3+http://polymath/backup /home/mav/restored_backup/
  
  On T the log is much shorter, which is unfortunate :-)
2009-03-11_05:44:13: Starting restore procedure for backup-restore (backup on 
aws). Restored to /home/mav/restored_backup/
2009-03-11_05:44:14: /usr/local/bin/duplicity restore --encrypt-key BD0CC45E 
--sign-key BD0CC45E --verbosity 9 s3+http://polymath/backup 
/home/mav/restored_backup/
Main action: restore
0 files exist on backend
Extracting backup chains from list of files: []
Collection Status
-----------------
Connecting with backend: BotoBackend
Archive dir: None

Found 0 backup chains without signatures.
No backup chains with active signatures found
No orphaned or incomplete backup sets found.
Last full backup date: none
Using temporary directory /tmp/duplicity-bq1sRB-tempdir
Traceback (most recent call last):
  File "/usr/local/bin/duplicity", line 589, in <module>
    with_tempdir(main)
  File "/usr/local/bin/duplicity", line 582, in with_tempdir
    fn()
  File "/usr/local/bin/duplicity", line 544, in main
    restore(col_stats)
  File "/usr/local/bin/duplicity", line 304, in restore
    restore_get_patched_rop_iter(col_stats)):
  File "/usr/local/bin/duplicity", line 321, in restore_get_patched_rop_iter
    backup_chain = col_stats.get_backup_chain_at_time(time)
  File "/usr/local/lib/python2.5/site-packages/duplicity/collections.py", line 
798, in get_backup_chain_at_time
    raise CollectionsError("No backup chains found")
CollectionsError: No backup chains found

2009-03-11_05:44:14: Restore complete

Now for the weird things: I've tried to download the entire content of
the Amazon bucket on T using s3cmd, like so:

    s3cmd get --recursive s3://polymath/backup ~/copy_of_remote_backup
    
if then I change the restore script to execute:
   /usr/local/bin/duplicity restore --encrypt-key BD0CC45E --sign-key BD0CC45E 
--verbosity 9 file://home/mav/copy_of_remote_backup /home/mav/restored_backup/

This fails with the same stack dump.

If I do the same on S (where duplicity is succesful) and execute the
same:
   /usr/local/bin/duplicity restore --encrypt-key BD0CC45E --sign-key BD0CC45E 
--verbosity 9 file://home/mav/copy_of_remote_backup /home/mav/restored_backup/
I get the same stack dump talking about no chains found.

Now, another strange thing I've noticed. If I do use s3cmd to list the
Amazon directory, I get

address@hidden:~> s3cmd  ls s3://polymath
                       DIR   s3://polymath/backup/
2009-03-09 18:36         0   s3://polymath/backup_$folder$

I don't know what the backup_$folder$ is, but listing
s3://polymath/backup/ seems to be ok and contains the duplicity files.

Is there a way to tell duplicity not to delete the files it downloads
through the backend? Or to stop before decripting/extracting them?
I'd like to compare those to what I get w/ s3cmd and see if there's a
difference between what's downloaded on S and what's downloaded on T.

Some answers to your questions are in the fllowing text.

> Maurizio Vitale wrote:
>> I'm trying to move my backup system to Amazon S3. I have backup and
>> restore working on the system I'm making backups from (OpenSuse 11.1).
>> 
>> But when I try to restore from anywhere else (a phisical machine running
>> Ubuntu 8.10 and a number of vmware machines running pretty much
>> everything, from Ubuntu 8.10 to OpenSuse 11.1 to Debian 5.0) I get stack
>> traces like the following (this is taken on the Ubuntu 8.10 physical 
>> machine): 
>> 
>> 2009-03-09_22:24:37: Starting restore procedure for thor (backup on aws). 
>> Restored to /home/mav/restored_backup/
>> Last full backup date: none
>> Traceback (most recent call last):
>>   File "/usr/bin/duplicity", line 482, in <module>
>>     with_tempdir(main)
>>   File "/usr/bin/duplicity", line 477, in with_tempdir
>>     fn()
>>   File "/usr/bin/duplicity", line 441, in main
>>     restore(col_stats)
>>   File "/usr/bin/duplicity", line 223, in restore
>>     restore_get_patched_rop_iter(col_stats)):
>>   File "/usr/bin/duplicity", line 238, in restore_get_patched_rop_iter
>>     backup_chain = col_stats.get_backup_chain_at_time(time)
>>   File "/usr/lib/python2.5/site-packages/duplicity/collections.py", line 
>> 717, in get_backup_chain_at_time
>>     raise CollectionsError("No backup chains found")
>> duplicity.collections.CollectionsError: No backup chains found
>> 2009-03-09_22:24:37: Restore complete
>> 
>> The backup is produced by duplicity 0.5.10. The restores are w/ 0.5.11
>> (except the one on the system being abcked up, which is 0.5.10).
>> 
>> The boto library vearies w/ the system. I have 1.6b, 1.6a, 1.3a and
>> 1.2a.
>> 
>> Any idea about what to look for?
>
> Make sure the backup bucket names match.

Using the same script on both machine, so I should be ok here.

> Make sure that the backup and restore command lines match w.r.t. options
> dealing with filenames (--short-filenames or --old-filenames).
>

Not using any of them

> The "Last full backup date: none" and "No backup chains found" tells me
> that there may not be any files in the bucket you selected.  Try one of
> the S3 access tools to list the bucket.

I can see the files with Firefox and s3cmd.

> If all else fails, show us the command lines for both restore and
> backup, run the restore with -v9 and dump the log.  If possible, make
> sure the version of boto is the same for backup and restore.  It has
> been known to differ between versions.  Since it is working on the
> OpenSuse system, my guess its a mismatch elsewhere.

Made two systems use the same version of boto. No changes.

> ...Ken
>
> _______________________________________________
> Duplicity-talk mailing list
> address@hidden
> http://lists.nongnu.org/mailman/listinfo/duplicity-talk

-- 
Maurizio Vitale
Polymath Solutions