duplicity-talk
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Duplicity-talk] decryption failure in duplicity replicate


From: zga9uhnq4g
Subject: Re: [Duplicity-talk] decryption failure in duplicity replicate
Date: Sat, 28 Aug 2021 14:05:12 -0700
User-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.13.0

Thanks again for your help and advise Edgar.  Following your advise I was finally able get the first (original) backup chain work with duplicity.  Fortunately, I only had to modify the manifest for the full backup, but it took me a while to figure out that I did have the problem you described with the manifests being cached locally. Once I used --name to create a new cache I was able to verify and restore from time 2020-12-12.

My next step was try to replicate from OneDrive to Google Drive.  It seemed to be working, and replicated all (I assume) the manifest and sigtar files, but when it got to replicating the first (200 MB) vol1 file it displayed the message "RedirectMissingLocation: Redirected but the response is missing a Location: header" 4 times, and then "Giving up after 5 attempts. RedirectMissingLocation: Redirected but the response is missing a Location:header" and stopped.  Searching the internet for those message, it appears to be a known problem, a compatibility issue between duplicity, google-api-python-client, and httplib2.  I checked the versions of those packages in my (Debian/Testing) environment:
# dpkg -l | fgrep httplib
ii  python3-google-auth-httplib2  0.0.4-2 all          Google Authentication Library: httplib2 transport ii  python3-httplib2              0.18.1-3 all          comprehensive HTTP client library written for Python3
# dpkg -l | fgrep google
ii  python3-google-auth           1.5.1-3 all          Google Authentication Library - Python 3.x ii  python3-google-auth-httplib2  0.0.4-2 all          Google Authentication Library: httplib2 transport ii  python3-google-auth-oauthlib  0.4.2-1 all          oauthlib integration with google-auth ii  python3-googleapi             1.7.11-4 all          Google APIs Client Library - Python 3.x and then based on a claimed solution (workaround) in https://stackoverflow.com/questions/59815620/gcloud-upload-httplib2-redirectmissinglocation-redirected-but-the-response-is-m, I ran "pip install httplib2==0.15.0" (I did not run "pip install google-api-python-client==1.6".  Pip warned me about not being able to uninstall httplib2 0.18.1 (because it is install as a debian package, not a pip package), but install httplib2 0.15.0 under /usr/local/lib/python3.9/dist-packages.  When I tried "duplicity replicate ..." again it was now able to replicate the vol1.difftar files and completed successfully and I was able to successfully verify (latest and --time 2020-12-12) and do a test restore from Google Drive.

Using pip to install an older version of httplib2 seems like a workaround, not a real fix.  I don't really understand which component (duplicity, google-api-python-client, or httplib2) has the problem, and whether there is a real fix available.  Does anyone else know how I can avoid the workaround of installing an 8 month old version of httplib2?

    thanks,
    Peter

On 2021-08-25 03:33, edgar.soldin--- via Duplicity-talk duplicity-talk-at-nongnu.org |duplicity-talk| wrote:
On 25.08.2021 10:26, zga9uhnq4g--- via Duplicity-talk wrote:
Okay, I tried re-encrypting the 2020-12-12 files and copying them back to OneDrive, but 
when I ran "duplicity verify --time 2020-12-12 ..." I got

     Local and Remote metadata are synchronized, no sync needed.
     Last full backup date: Wed Aug 11 02:01:44 2021
     Invalid data - SHA1 hash mismatch for file:
      duplicity-full.20201212T204209Z.vol1.difftar.gpg
      Calculated hash: bcac07d8b88aeff2b928090c35db072fe1444f47
      Manifest hash: 0529e8d2384b24dec4d771f50dacca7d585188f8

So must have done something wrong.  The command I used to reencrypt was

     gpg2 --symmetric --cipher-algo AES256 $file

Do I need to give more or different option to gpg?
no, the re-encryption seems to have enabled duplicity to decrypt the files now.

the error comes from duplicity now as it correctly sees that the volumes were 
modified after their first creation. it's possible that adding 
'--ignore-errors' will not fail on the error but that should only be a last 
resort, as it'll silently ignore other errors too, that might theoretically 
there. nope, just tested, fails still.

just did some local tests and it looks like the hash is saved in the manifest 
file under
"
Volume 1:
     StartingPath   ...
     EndingPath     ...
     Hash SHA1 df901ba65335d73e6e9724c4fc9ed79adda4dfac
"
it should contain 0529e8d2384b24dec4d771f50dacca7d585188f8 now, replace it with 
0529e8d2384b24dec4d771f50dacca7d585188f8, encrypt the new manifest and that 
should get rid of the failed checksum in duplicity. you may need to do this for 
all volumes obviously. 'sha1sum' can help you.

if that does not work instantly it might be because duplicity caches manifests 
from remote. delete the archive dir cache for the repo to force duplicity to 
become aware of the change. easiest is to provide a  '--name foobar', then you 
will find the folder under .cache/duplicity/foobar . delete it after you did 
changes to your test repo and run duplicity only after and it will be recreated.

for your tests you may just place the files in a folder locally and run "duplicity 
verify file:///absolute/local/folder". you can leave out time if you just place the 
first full (and later incrementals) there as duplicity will test the one chain it should 
find.

to make it maybe even more easier/complicated you may try working with the unencrypted 
files by putting them in a folder and run duplicity like "duplicity 
collection-status --no-encryption --no-compression file:///absolute/local/folder" . 
that might probably fail though due to the checksums from above not being modified too.

good luck ..ede/duply.net

_______________________________________________
Duplicity-talk mailing list
Duplicity-talk@nongnu.org
https://lists.nongnu.org/mailman/listinfo/duplicity-talk




reply via email to

[Prev in Thread] Current Thread [Next in Thread]