Re: [Duplicity-talk] Failures w/ S3 backend

[Kenneth Loafman]

Glad you found it.  Looks like it may be a boto issue.

I use ntp daemon for setting time.  Lots of experience with funny
problems cause by time drift, especially when using tools like 'make'.

I can easily imagine boto or S3 would have some sanity check in it to
make sure you don't load new files 'older' than the ones already there.

Will keep this in mind for the next time around.  Gotta build a FAQ some
day soon.

...Thanks,
...Ken

Maurizio Vitale wrote:
> 
> ok, great news: I was incredibly lucky in that  electrical problems at
> the office
> forced me to rearrange equipment a week or so ago.
> 
> Net result is that the physical machine I was planning to use for the
> experiments
> remained plugged off for a week. At power on it shared one caracteristics
> with all my vmware machines and sure enough restores were failing.
> 
> The common trait is the system date: if it is wrong all sort of nasty
> things happens.
> [btw, I'm talking about time being wrong by days, haven't checked with
> smaller errors]
> 
> We have seen the stack backtrace I posted a while ago and this.
> Here the situation is that duplicity exits with an error message saying
> that there're no
> backup chains. Problem is there is a valid backup set and it is
> something in the duplicity interface
> with boto (or in boto itself) that is wrong when time is skewed.
> 
> Once the system time was set properly, the physical machine restored the
> backup just fine.
> I went back to the vmware machines. Sure enough the time was wrong (btw,
> how do people
> force the time to be set properly when exiting from vmware suspend mode?).
> Again, setting the right time solved the problem and the vmware machine
> was able to
> restore everything.
> 
> Note that copying files with s3cmd was succesful on the VM with the
> wrong time, so it doesn't seem
> to be intrinsic in the S3 protocol, but rather something in the way boto
> handles it.
> 
> If this cannot be fixed easily, it is probably worthwhile to put a note
> about it in the documentation.
> Might save some people's day (especially if they're actually trying to
> restore after a crash as opposed
> to me just testing my setup)
> 
> Thanks to all who helped.
> 
>     Maurizio
> 
> On Mar 19, 2009, at 10:37 AM, Maurizio Vitale wrote:
> 
>>
>> Haven't had time to investigate much more, but here a few interesting
>> new facts.
>>
>> Scenario:
>>     - backup from OpenSuse 11.1, 64 bit, called S (for source)
>>         - duplicity 0.5.11
>>         - boto 1.6b
>>     - restore on Ubuntu 8.10, 64 bit running in a vmware VM, called T
>> (for target)
>>         - duplicity 0.5.10
>>         - boto 1.6b
>>
>>     backup on Amazon S3, everything uses the same backup/restore scripts
>>
>> What works:
>>     backup on S, restore on S
>>     backup on T, restore on S
>>     backup on S, restore on T (but not for all backups: my main
>> workspace fails, a smaller directory
>>                                                          is ok. Size
>> doesn't seem to matter, though: I tried to reduce the size
>>                                                          of (a copy
>> of) the main workspace and it kept failing
>>
>>     and now the new and interesting thing:
>>           backup on S (full workspace)
>>       copy of all files from S to T (using s3cmd)
>>       restore from the local copy (using
>> file:///path_to_downloaded_files)
>>     much to my surprise, this works.
>>
>> Any idea? It seems like the combination of duplicity/boto when run on
>> my vmware machines fail to
>> recognize the files as valid. Possibly they get corrupted while in
>> flight.
>>
>> My next test will be to go to a physical machine (different from S)
>> and see if I can restore.
>>
>> Any other suggestions?
>> Anybody tried to restore on vmware?
>>
>> On Mar 10, 2009, at 2:00 PM, Maurizio Vitale wrote:
>>
>>>
>>> I'm trying to move my backup system to Amazon S3. I have backup and
>>> restore working on the system I'm making backups from (OpenSuse 11.1).
>>>
>>> But when I try to restore from anywhere else (a phisical machine running
>>> Ubuntu 8.10 and a number of vmware machines running pretty much
>>> everything, from Ubuntu 8.10 to OpenSuse 11.1 to Debian 5.0) I get stack
>>> traces like the following (this is taken on the Ubuntu 8.10 physical
>>> machine):
>>>
>>> 2009-03-09_22:24:37: Starting restore procedure for thor (backup on
>>> aws). Restored to /home/mav/restored_backup/
>>> Last full backup date: none
>>> Traceback (most recent call last):
>>>   File "/usr/bin/duplicity", line 482, in <module>
>>>     with_tempdir(main)
>>>   File "/usr/bin/duplicity", line 477, in with_tempdir
>>>     fn()
>>>   File "/usr/bin/duplicity", line 441, in main
>>>     restore(col_stats)
>>>   File "/usr/bin/duplicity", line 223, in restore
>>>     restore_get_patched_rop_iter(col_stats)):
>>>   File "/usr/bin/duplicity", line 238, in restore_get_patched_rop_iter
>>>     backup_chain = col_stats.get_backup_chain_at_time(time)
>>>   File "/usr/lib/python2.5/site-packages/duplicity/collections.py",
>>> line 717, in get_backup_chain_at_time
>>>     raise CollectionsError("No backup chains found")
>>> duplicity.collections.CollectionsError: No backup chains found
>>> 2009-03-09_22:24:37: Restore complete
>>>
>>> The backup is produced by duplicity 0.5.10. The restores are w/ 0.5.11
>>> (except the one on the system being abcked up, which is 0.5.10).
>>>
>>> The boto library vearies w/ the system. I have 1.6b, 1.6a, 1.3a and
>>> 1.2a.
>>>
>>> Any idea about what to look for?
>>>
>>> TIA,
>>>
>>>         Maurizio
>>>
>>>
>>>
>>> _______________________________________________
>>> Duplicity-talk mailing list
>>> address@hidden
>>> http://lists.nongnu.org/mailman/listinfo/duplicity-talk
>>
>>
>>
>> _______________________________________________
>> Duplicity-talk mailing list
>> address@hidden
>> http://lists.nongnu.org/mailman/listinfo/duplicity-talk
> 
> 
> 
> _______________________________________________
> Duplicity-talk mailing list
> address@hidden
> http://lists.nongnu.org/mailman/listinfo/duplicity-talk
>

signature.asc
Description: OpenPGP digital signature