Re: [Duplicity-talk] problem with restores from S3

[Russell Clemings]

Me again ...

Any idea what the problem might be on this one? This is from my backup
script. It worked fine last night, works fine right now, but this
morning (shortly after I wrote the message below) I got failures.

Does it look like just a transitory connection issue or something
worse? As I said, it's fine now, but I hate to leave these things
unresolved.

Download 
s3+http://BUCKETNAME/duplicity-full-signatures.2008-10-04T02:20:10-04:00.sigtar.gpg
failed (attempt #1)
Download 
s3+http://BUCKETNAME/duplicity-full-signatures.2008-10-04T02:20:10-04:00.sigtar.gpg
failed (attempt #2)
Download 
s3+http://BUCKETNAME/duplicity-full-signatures.2008-10-04T02:20:10-04:00.sigtar.gpg
failed (attempt #3)
Download 
s3+http://BUCKETNAME/duplicity-full-signatures.2008-10-04T02:20:10-04:00.sigtar.gpg
failed (attempt #4)
Download 
s3+http://BUCKETNAME/duplicity-full-signatures.2008-10-04T02:20:10-04:00.sigtar.gpg
failed (attempt #5)
Giving up trying to download
s3+http://BUCKETNAME/duplicity-full-signatures.2008-10-04T02:20:10-04:00.sigtar.gpg
after 5 attempts
Traceback (most recent call last):
 File "/usr/local/bin/duplicity", line 522, in ?
   with_tempdir(main)
 File "/usr/local/bin/duplicity", line 516, in with_tempdir
   fn()
 File "/usr/local/bin/duplicity", line 509, in main
   incremental_backup(sig_chain)
 File "/usr/local/bin/duplicity", line 239, in incremental_backup
   sig_chain.get_fileobjs(), new_sig_outfp)
 File "/usr/local/lib/python2.4/site-packages/duplicity/collections.py",
line 384, in get_fileobjs
   return map(filename_to_fileobj, [self.fullsig] + self.inclist)
 File "/usr/local/lib/python2.4/site-packages/duplicity/backend.py",
line 279, in get_fileobj_read
   self.get(filename, tdp)
 File 
"/usr/local/lib/python2.4/site-packages/duplicity/backends/botobackend.py",
line 179, in get
   raise BackendException("Error downloading %s/%s" %
(self.straight_url, remote_filename))
duplicity.errors.BackendException: Error downloading
s3+http://BUCKETNAME/duplicity-full-signatures.2008-10-04T02:20:10-04:00.sigtar.gpg

rac

On 10/9/08, Russell Clemings <address@hidden> wrote:
> That works, thanks. My confusion lay in a misreading of the man page:
>
>  "Now suppose we accidentally delete /home/me and want to restore it
>  the way it was at the time of last backup:
>     duplicity scp://address@hidden/some_dir /home/me
>  Duplicity enters restore mode because the URL comes before the local
>  directory. If we wanted to restore just the file "Mail/article" in
>  /home/me as it was three days ago into /home/me/restored_file:
>     duplicity -t 3D --file-to-restore Mail/article
>  scp://address@hidden/some_dir /home/me/restored_file"
>
>  I took the first to be the command you use to restore a whole
>  directory, and the second to restore a single file. I guess the first
>  is actually used only if you want to overwrite the directory you're
>  restoring. I'm not brave enough for that.
>
>  For the archives:
>
>  # Restore entire /root directory to new /root1 directory
>
> /usr/local/bin/duplicity -v9 --encrypt-key=${GPG_KEY} \
>  --sign-key=${GPG_KEY} --file-to-restore root s3+http://BUCKETNAME/ /root1
>
>
> # Restore single file "filename.txt" to new location /root/filename.txt1
>
> /usr/local/bin/duplicity -v9 --encrypt-key=${GPG_KEY} \
>
> --sign-key=${GPG_KEY} --file-to-restore root/filename.txt
>  s3+http://BUCKETNAME/ \
>  /root/filename.txt1
>
>  # Restore /root by overwriting from last backup
>
> /usr/local/bin/duplicity -v9 --encrypt-key=${GPG_KEY}
>  --sign-key=${GPG_KEY} \ s3+http://BUCKETNAME/ /root
>
>
> rac
>
>
>  On 10/9/08, Tim Riemenschneider <address@hidden> wrote:
>  > That's probably a misunderstanding in the command-line-arguments:
>  >
>  >  Russell Clemings schrieb:
>  >
>  > > address@hidden s3backup]# cat backup.sh
>  >  > #!/bin/sh
>  >  > export AWS_ACCESS_KEY_ID=XXXXXXXXX
>  >  > export AWS_SECRET_ACCESS_KEY=XXXXXXXXX
>  >  > export PASSPHRASE=XXXXXXXXX
>  >  > export GPG_KEY=XXXXXXXXX
>  >  > SOURCE=/
>  >  > DEST=s3+http://BUCKETNAME
>  >  >
>  >
>  > ^^^^^^^^^^^^
>  >  Your destiniation is the root of your bucket.
>  >
>  > > 
> http://s3.amazonaws.com/BUCKETNAME/duplicity-new-signatures.2008-10-08T01:51:13-04:00.to.2008-10-08T21:58:33-04:00.sigtar.gpg
>  >  > 
> http://s3.amazonaws.com/BUCKETNAME/duplicity-inc.2008-10-08T01:51:13-04:00.to.2008-10-08T21:58:33-04:00.manifest.gpg
>  >  > 
> http://s3.amazonaws.com/BUCKETNAME/duplicity-inc.2008-10-08T01:51:13-04:00.to.2008-10-08T21:58:33-04:00.vol1.difftar.gpg
>  >  > 
> http://s3.amazonaws.com/BUCKETNAME/duplicity-inc.2008-10-08T01:51:13-04:00.to.2008-10-08T21:58:33-04:00.vol2.difftar.gpg
>  >  >
>  >  >
>  >
>  > The files are directly in the root of your bucket.
>  >
>  > > Now, here's the restore script:
>  >  > address@hidden s3backup]# cat restore.sh
>  >  > #!/bin/sh
>  >  > export AWS_ACCESS_KEY_ID=XXXXXXXXX
>  >  > export AWS_SECRET_ACCESS_KEY=XXXXXXXXX
>  >  > export PASSPHRASE=XXXXXXXXX
>  >  > export GPG_KEY=XXXXXXXXX
>  >  > /usr/local/bin/duplicity -v9 --encrypt-key=${GPG_KEY}
>  >  > --sign-key=${GPG_KEY} s3+http://BUCKETNAME/root /root1
>  >  >
>  >
>  > You try to restore from a directory root inside your bucket, that does
>  >  not exist.
>  >
>  > > address@hidden s3backup]# ./restore.sh
>  >  > Main action: restore
>  >  > 0 files exist on backend
>  >  >
>  >
>  > .... and duplicity says so.
>  >
>  >  What you need is:
>  >
>  >
>  >  /usr/local/bin/duplicity -v9 --encrypt-key=${GPG_KEY}
>  >
>  > --sign-key=${GPG_KEY} --file-to-restore root s3+http://BUCKETNAME/ /root1
>  >
>  >  cu
>  >  Tim
>  >
>