RE: [Duplicity-talk] SHA1 hash mismatch - need restore in any case, a bit urgent

[Igor Balic]

Yeah, I've traid that too, even changing the library files to remove 
exceptions, but some other assert always comes up. I guess some core 
modification is needed.
Duplicity really should have a disaster recovery mode/tool.

Thanks for the info on snapshots, I haven't managed to pull the data out of 
them yet, but still trying...

Igor


===
From: address@hidden [mailto:address@hidden On Behalf Of Kenneth Loafman
Sent: Saturday, April 09, 2011 12:36 AM
To: Discussion of the backup program duplicity
Subject: Re: [Duplicity-talk] SHA1 hash mismatch - need restore in any case, a 
bit urgent

I have tried turning off log.FatalError (comment out last two lines) and 
asserts in duplicity (python -O duplicity), however, I can't get Python to 
ignore asserts in library calls.  If you know how to do this, let me know and I 
think you might be able to execute duplicity in full ignore-error mode.

snapshot - full file in block, no parts to append.
diff - rdiff of corresponding snapshot
multivol_snapshot - numbered 64k blocks that need to be concatenated.
multivol_diff - rdiff of corresponding 64k block in multivol_snapshot.

Essentially, apply the diff to the snapshot and you have the incremental file.  
The multivol_diff should be applied to the corresponding multivol_snapshot 
based on the block number.  YMMV.  I've never tried this before, but it should 
work.

...Ken
On Fri, Apr 8, 2011 at 4:42 PM, Igor Balic <address@hidden> wrote:
A bit more problems with increments, can anyone please give me an example how 
to apply increments? There are 4 dirs:

- snapshot
- multivol_diff
- multivol_snapshot
- diff

how to apply those to full backup and in what order?


Igor

-----Original Message-----
From: address@hidden [mailto:address@hidden On Behalf Of Igor Balic
Sent: Friday, April 08, 2011 11:26 PM
To: 'Discussion of the backup program duplicity'
Subject: RE: [Duplicity-talk] SHA1 hash mismatch - need restore in any case, a 
bit urgent

Hi Ken,

I will hold on to files, but need to sort this mess out first.
Is there maybe an easy way to get duplicity to do this function, i.e. skip the 
corrupt archive(s) and extract what it can from others? Seems like a pretty 
straight forward job. I know it isn't a clean way to do backups, but when all 
else fails, anything is better than nothing...


Igor



====
From: address@hidden [mailto:address@hidden On Behalf Of Kenneth Loafman
Sent: Friday, April 08, 2011 11:18 PM
To: Discussion of the backup program duplicity
Subject: Re: [Duplicity-talk] SHA1 hash mismatch - need restore in any case, a 
bit urgent

Hi Igor,

Your plan sounds like it should work.  That is essentially what duplicity would 
do internally.

Later, after you get past this problem, please get back to us.  I'd like to try 
to understand how the truncated file got built, and why it got truncated.  What 
I need to know is:
-- how you performed the backup?
-- is the file:// URL a mounted filesystem?
-- if mounted, what type of mount?
-- what version of duplicity are you using?

Please hold on to the backups if possible.

...Thanks,
...Ken
On Fri, Apr 8, 2011 at 3:01 PM, Igor Balic <address@hidden> wrote:
Please confirm if this plan is OK:

- I extract all full volumes in dir one
- I extract all increment volumes in dir two
- I use rdiff with option to overwrite all changed files, with source one to 
two . This will overwrite all changed "chunks"
- I use a script that is yet to be written to convert multivol directories back 
into files and in correct dirs:
    - cat all chunks into file with name of dirname.tmp
    - move this file one dir up
    - remove dir
    - move file from dirname.tmp to dirname
    - after all are files assembled, move them to snapshot dir


But this will keep all files that got deleted from full to inc backup? Is 
deleted files list written somewhere?


Igor



-----Original Message-----
From: address@hidden [mailto:address@hidden On Behalf Of Michael Terry
Sent: Friday, April 08, 2011 9:29 PM
To: Discussion of the backup program duplicity
Cc: Igor Balic
Subject: Re: [Duplicity-talk] SHA1 hash mismatch - need restore in any case, a 
bit urgent

So, if deleting your ~/.cache/duplicity like Clemens suggested didn't
work, I have written some instructions for how to recover files
manually from volumes.  It's not pretty, but maybe it would help if
you're desperate.  It also should let you get around the issue of the
first volume being bad (assuming that volume didn't hold the data you
cared about).

https://live.gnome.org/DejaDup/Help/Restore/WorstCase#Restoring_by_Hand

-mt

On 8 April 2011 15:18, Igor Balic <address@hidden> wrote:
> I have inspected files again, more carefully, volume 1 (of ~500) of full 
> backup is corrupted (unexpected end of archive when gunzipping manually), 
> it's 10MB long, but it should be 50MB. :(
> Is there any way to skip files in this volume and just proceed with others? 
> It's all non-ecrypted.
>
> Igor
>
>
> -----Original Message-----
> From: address@hidden [mailto:address@hidden On Behalf Of Clemens Eisserer
> Sent: Friday, April 08, 2011 9:04 PM
> To: Discussion of the backup program duplicity
> Subject: Re: [Duplicity-talk] SHA1 hash mismatch - need restore in any case, 
> a bit urgent
>
> Hi igor,
>
> I saw the same error when I tried to restore a backup of mine.
> In my case I just deleted all cached metdata, and it suddenly worked.
>
> Although I am a bit unsure how much I can trust duplicity now :/
>
> Good luck, Clemens
>
> 2011/4/8 Igor Balic <address@hidden>:
>> Hi,
>>
>> sorry for urgency. We have a duplicity full backup and one increment.
>> Backups are not encrypted. Output is:
>>
>> duplicity restore --force  --no-encryption --file-to-restore /
>> file:///duplicity/___/home2    /root/res
>> Local and Remote metadata are synchronized, no sync needed.
>> Last full backup date: Tue Feb  8 19:39:53 2011
>> Invalid data - SHA1 hash mismatch:
>> Calculated hash: 3ca1ea655bdcc978ddb2711bed013e30bd880a03
>> Manifest hash: f46cab5c38e1ae506133c8f568e5773e0f5f7626
>>
>> I need a way to restore files, despite this error, --force doesn't help,
>> neither does using --file-to-restore, or without it.
>>
>> Backups went OK, without error, file sizes seem OK, no errors when backup
>> was made, both machines are 64 bit.
>> I just need to somehow override this error, modifying duplicity executable
>> is OK.
>>
>> I can pay for fast solution, since I really need it... No usable solution
>> found in archives.
>>
>> Igor
>>
>>
>> _______________________________________________
>> Duplicity-talk mailing list
>> address@hidden
>> http://lists.nongnu.org/mailman/listinfo/duplicity-talk
>>
>
> _______________________________________________
> Duplicity-talk mailing list
> address@hidden
> http://lists.nongnu.org/mailman/listinfo/duplicity-talk
>
>
> _______________________________________________
> Duplicity-talk mailing list
> address@hidden
> http://lists.nongnu.org/mailman/listinfo/duplicity-talk
>


_______________________________________________
Duplicity-talk mailing list
address@hidden
http://lists.nongnu.org/mailman/listinfo/duplicity-talk



_______________________________________________
Duplicity-talk mailing list
address@hidden
http://lists.nongnu.org/mailman/listinfo/duplicity-talk


_______________________________________________
Duplicity-talk mailing list
address@hidden
http://lists.nongnu.org/mailman/listinfo/duplicity-talk