Re: [Duplicity-talk] decryption failure in duplicity replicate

[zga9uhnq4g]

    Thanks for suggestions Edgar.
    
    I get the same error "gpg: decryption failed: Bad session key" when
    trying verify/restore my backup from 2020-12-12.
    
    Following the info in the link you mentioned, I tried manually
    decrypting via gpg.  I found that I could decrypt all the
    incremental backups in the chain (at least all the ones I tried),
    but decrypting the original full backup failed with  "gpg:
    decryption failed: Bad session key".  Thinking back, I think I know
    what happened.  That first full backup was executed interactively
    from the command-line, and I got prompted for the passphrase as
    copy/pasted it in.  All the backup after that were executed from my
    backup script (that I schedule to run at 3:30, and there I use the
    PASSPHRASE environment variable.  I was even able to confirm that
    (sort of) by looking in my .bash_history file.  I had just about
    resigned myself to the fact that the first backup chain was useless,
    but I just came back after a couple hours, had an idea, and figured
    out the passphrase for that original full backup.  As is common, my
    passphrase is words separated by spaces (e.g. the quick brown dog
    jumps), so when I set PASSPHRASE (for some reason I don't remember)
    I used backslashes to escape the spaces (e.g. PASSPHRASE=the\ quick\
    brown\ dog\ jumps).  I discovered that when I run "gpg ... -decrypt
    ..." and when it prompts for the passphrase I type the passprhase
    with the backslashes it successfully decrypts.
    
    So, as Edgar suggested, my duplicity backup does have two different
    passphrases, a first for the original full backup, and a second for
    everything else.  Is there anyway to get duplicity to use the two
    passphrases correctly, or is my only option to manually decrypt the
    the 4 files (1 manifest, 2 difftars, and 1 sigtar) with the first
    passphrase and re-encrypt them with the second passphrase and then
    replace the 4 original files with the re-encrypted ones?
    
    
    Everything below describes my attempt to manually copy the backup
    from  Microsoft OneDrive to Google Drive as Edgar suggested and get
    duplicity to recognize it.  It was unsuccessful, and the behavior
    was strange and surprising, so I'm only describing it for folks that
    are curious.  Feel free to skip this if your not interested, as it
    seems like a dead end.
    
    I copied the entire backup folder from Microsoft OneDrive to Google
    Drive by dragging the folder from Windows Explorer in the Google
    Drive web page (took a couple hours), but when I tried "duplicity
    verify" pointing at the newly copied Google Drive folder, it said:
> Local and Remote metadata are
>         synchronized, no sync needed.
>         Last full backup date: none
>     
    followed by a Traceback and ending with the exception:
> duplicity.dup_collections.CollectionsError:
>         No backup chains found
>     
    The strange thing is that it created a new empty folder in Google
    Drive, with the same name as the newly copied folder and right next
    to it.  I also tried "duplicity collecton-status" and it said:
> Collection Status
>         -----------------
>         Connecting with backend: BackendWrapper
>         Archive dir:
>         /root/.cache/duplicity/f509c410dcf8a9acb918e98cb707d4da
>         
>         Found 0 secondary backup chains.
>         No backup chains with active signatures found
>         No orphaned or incomplete backup sets found.
>       
    and also created a new empty folder if it didn't already exist. 
    Looking in the Google Drive web UI at the two folders, one created
    by manually copying a folder into the Google Drive web UI, and one
    created by duplicity, I can't see any differences.  Very strange. 
    I'll go back to trying to get "replicate" to work.
    
        thanks,
        Peter
    
    On 2021-08-23 04:40, edgar.soldin--- via Duplicity-talk
    duplicity-talk-at-nongnu.org |duplicity-talk| wrote:
> hey Peter,
>
> On 23.08.2021 07:47, zga9uhnq4g--- via Duplicity-talk wrote:
> > Thanks for the hints Ken.  I did find lots of info talking about changes between gpg1 and gpg2, and when I "duplicity" to the search term, I found a bunch of duplicity-specific stuff basically saying the same thing, including https://askubuntu.com/questions/1057627/duplicity-fails-with-bad-session-key-error with an answer from Edgar.  I followed the links he mentioned, and tried the workarounds described in https://superuser.com/questions/984977/duplicity-restore-failing-no-secret-key, but they didn't help, so either I didn't do them correctly or my problem is different.
> >
> > I'm skeptical that it could be a gpg1 versus gpg2 issue, since I started using duplicity 2020-12-12 and based on /var/log/apt/history.log, I had the gnupg and gpg* packages at version 2.2.20-1.  Later, on 2021-06-25, I upgraded these packages to version 2.2.27-2, but AFACT I never used a version earlier than 2.2.20.
> >       
> the solution, issue/workaround described is an upgrade from gpg 2.0 to 2.1+ . but the error actually just says "wrong or no password given"
>
> my guess is that you managed to modify the password used to gpg encrypt your backup between chains.
>
> what is the exact error when trying to restore your backup from 12.Dec.2020 ?
>
> > I did some further testing using both "duplicity verify" and "duplicity restore" for the "latest" backup which succeed, and then using "--time 2020-12-12" to access my first backup which failed with the "Bad session key" error from gpg.  I then used --time to try verify/restore backups from 2021-08-01, 2021-07-01, 2021-06-01, etc., then narrowed failure down to a single day.  It appears that I can verify/restore all backups after 2021-02-10, but verify/restore fail for the backups on or before 2021-02-09.  I just figured out that I added "--full-if-older-than 1M" to the duplicity backup command in my backup script on 2021-02-08, so it took effect on the backup on 2021-02-09. so "duplicity collection status" shows
> >
> >     Last full backup date: Wed Aug 11 02:01:44 2021
> >     Collection Status
> >     -----------------
> >     Connecting with backend: BackendWrapper
> >     Archive dir: /root/.cache/duplicity/770219317cf70c5a3236e3cb0d9a212c
> >
> >     Found 7 secondary backup chains.
> >     Secondary chain 1 of 7:
> >     -------------------------
> >     Chain start time: Sat Dec 12 12:42:09 2020
> >     Chain end time: Mon Feb  8 03:31:07 2021
> >     Number of contained backup sets: 57
> >     Total number of contained volumes: 58
> >      Type of backup set:                            Time:      Num volumes:
> >                     Full         Sat Dec 12 12:42:09 2020                 2
> >              Incremental         Mon Dec 14 03:30:55 2020                 1
> >                       ...
> >              Incremental         Sun Feb  7 03:30:51 2021                 1
> >              Incremental         Mon Feb  8 03:31:07 2021                 1
> >     -------------------------
> >
> >     Secondary chain 2 of 7:
> >     -------------------------
> >     Chain start time: Tue Feb  9 03:31:06 2021
> >     Chain end time: Wed Mar 10 03:31:24 2021
> >     Number of contained backup sets: 30
> >     Total number of contained volumes: 31
> >      Type of backup set:                            Time:      Num volumes:
> >                     Full         Tue Feb  9 03:31:06 2021                 2
> >              Incremental         Wed Feb 10 03:31:06 2021                 1
> >     ...
> >
> > Since, as you can see my backups run at 3:30, specifying "--time 2021-02-09" (as I did in the testing I described above) is actually accessing the backup set  from 2021-02-08, so my guess is that any access to the first backup chain (Secondary chain 1 of 7)  is producing the "Bad session key" error from gpg.
> >       
> sounds reasonable
>
> > So now my real question is, is there any way to replicate my backup, but skip secondary chain 1 of 7.  The man page says "When /--time/ time is given, only backup sets older than time will be replicated", which seems like the opposite of what I need.  Is my only option to run "duplicity remove-older-than 2021-02-09" and then do the replicate?
> >       
> i am not really sure why replicate as functionality was added to duplicity, especially that way (recompressing,reencrypting), but what you can easily do is just move/copy the files on the backup repo to another backend and change the target for duplicity (use e.g. rclone).
>
> the backup files are named pretty conclusively, so sorting them by name should allow you to identify chains/backups. when you did that you can e.g. simply move the "broken" chain into another folder, out of the way and replicate should magically work. but again, no need to replicate at all if you want to keep encraption/compression as is. simply copy the files and run a verify against the new backup repo after and you are all set!
>
> > I would also still be interested in any suggestions for how I can make the backups from 2021-02-09 and before accessible again.
> >       
> as asked above.  what's the exact error? if it is still "gpg: decryption failed: Bad session key" then you could try to find out if it's a duplicity bug or gpg issue (wrong passphrase, passphrase not routed to process) by fetching a gpg volume from the offending chain and simply trying to decrypt manually via gpg to /tmp or such just to pinpoint the issue. see https://wiki.gnome.org/Apps/DejaDup/Help/Restore/WorstCase#Restoring_by_Hand
>
> good luck.. ede/duply.net
>
> _______________________________________________
> Duplicity-talk mailing list
> Duplicity-talk@nongnu.org
> https://lists.nongnu.org/mailman/listinfo/duplicity-talk
>