Re: [Duplicity-talk] Failures w/ S3 backend

[Maurizio Vitale]

ok, great news: I was incredibly lucky in that  electrical problems  
at the office
forced me to rearrange equipment a week or so ago.

Net result is that the physical machine I was planning to use for the  
experiments
remained plugged off for a week. At power on it shared one  
caracteristics
with all my vmware machines and sure enough restores were failing.

The common trait is the system date: if it is wrong all sort of nasty  
things happens.
[btw, I'm talking about time being wrong by days, haven't checked  
with smaller errors]

We have seen the stack backtrace I posted a while ago and this.
Here the situation is that duplicity exits with an error message  
saying that there're no
backup chains. Problem is there is a valid backup set and it is  
something in the duplicity interface
with boto (or in boto itself) that is wrong when time is skewed.

Once the system time was set properly, the physical machine restored  
the backup just fine.
I went back to the vmware machines. Sure enough the time was wrong  
(btw, how do people
force the time to be set properly when exiting from vmware suspend  
mode?).
Again, setting the right time solved the problem and the vmware  
machine was able to
restore everything.

Note that copying files with s3cmd was succesful on the VM with the  
wrong time, so it doesn't seem
to be intrinsic in the S3 protocol, but rather something in the way  
boto handles it.

If this cannot be fixed easily, it is probably worthwhile to put a  
note about it in the documentation.
Might save some people's day (especially if they're actually trying  
to restore after a crash as opposed
to me just testing my setup)

Thanks to all who helped.

        Maurizio

On Mar 19, 2009, at 10:37 AM, Maurizio Vitale wrote:

> Haven't had time to investigate much more, but here a few  
> interesting new facts.
>
> Scenario:
>         - backup from OpenSuse 11.1, 64 bit, called S (for source)
>                 - duplicity 0.5.11
>                 - boto 1.6b
> 	- restore on Ubuntu 8.10, 64 bit running in a vmware VM, called T  
> (for target)
>                 - duplicity 0.5.10
>                 - boto 1.6b
>
>         backup on Amazon S3, everything uses the same backup/restore scripts
>
> What works:
>         backup on S, restore on S
>         backup on T, restore on S
> 	backup on S, restore on T (but not for all backups: my main  
> workspace fails, a smaller directory
>                                                          is ok.  
> Size doesn't seem to matter, though: I tried to reduce the size
>                                                          of (a copy  
> of) the main workspace and it kept failing
>
>         and now the new and interesting thing:
>           backup on S (full workspace)
>           copy of all files from S to T (using s3cmd)
> 	  restore from the local copy (using file:/// 
> path_to_downloaded_files)
>         much to my surprise, this works.
>
> Any idea? It seems like the combination of duplicity/boto when run  
> on my vmware machines fail to
> recognize the files as valid. Possibly they get corrupted while in  
> flight.
>
> My next test will be to go to a physical machine (different from S)  
> and see if I can restore.
>
> Any other suggestions?
> Anybody tried to restore on vmware?
>
> On Mar 10, 2009, at 2:00 PM, Maurizio Vitale wrote:
>
> > I'm trying to move my backup system to Amazon S3. I have backup and
> > restore working on the system I'm making backups from (OpenSuse  
> > 11.1).
> >
> > But when I try to restore from anywhere else (a phisical machine  
> > running
> > Ubuntu 8.10 and a number of vmware machines running pretty much
> > everything, from Ubuntu 8.10 to OpenSuse 11.1 to Debian 5.0) I get  
> > stack
> > traces like the following (this is taken on the Ubuntu 8.10  
> > physical machine):
> >
> > 2009-03-09_22:24:37: Starting restore procedure for thor (backup  
> > on aws). Restored to /home/mav/restored_backup/
> > Last full backup date: none
> > Traceback (most recent call last):
> >   File "/usr/bin/duplicity", line 482, in <module>
> >     with_tempdir(main)
> >   File "/usr/bin/duplicity", line 477, in with_tempdir
> >     fn()
> >   File "/usr/bin/duplicity", line 441, in main
> >     restore(col_stats)
> >   File "/usr/bin/duplicity", line 223, in restore
> >     restore_get_patched_rop_iter(col_stats)):
> >   File "/usr/bin/duplicity", line 238, in  
> > restore_get_patched_rop_iter
> >     backup_chain = col_stats.get_backup_chain_at_time(time)
> >   File "/usr/lib/python2.5/site-packages/duplicity/ 
> > collections.py", line 717, in get_backup_chain_at_time
> >     raise CollectionsError("No backup chains found")
> > duplicity.collections.CollectionsError: No backup chains found
> > 2009-03-09_22:24:37: Restore complete
> >
> > The backup is produced by duplicity 0.5.10. The restores are w/  
> > 0.5.11
> > (except the one on the system being abcked up, which is 0.5.10).
> >
> > The boto library vearies w/ the system. I have 1.6b, 1.6a, 1.3a and
> > 1.2a.
> >
> > Any idea about what to look for?
> >
> > TIA,
> >
> >         Maurizio
> >
> >
> >
> > _______________________________________________
> > Duplicity-talk mailing list
> > address@hidden
> > http://lists.nongnu.org/mailman/listinfo/duplicity-talk
>
>
>
> _______________________________________________
> Duplicity-talk mailing list
> address@hidden
> http://lists.nongnu.org/mailman/listinfo/duplicity-talk