Re: [Duplicity-talk] assertion error while backing up

[Kenneth Loafman]

The fix you suggested was released today.  All tests pass so no breakage at all.

Thanks for the fix.

...Ken

On Tue, Feb 23, 2016 at 5:28 AM, Kenneth Loafman <address@hidden> wrote:

Thanks for the catch on the lowercasing.  I'm looking into why this is done, other than normalization.  I can understand it on some Mac/Windows filesystems where they are case-insensitive but case-preserving.  On Linux every FS I know of is case-sensitive.

On Mon, Feb 22, 2016 at 10:34 PM, Mark Grandi <address@hidden> wrote:

On Feb 15, 2016, at 2:20 PM, Mark Grandi <address@hidden> wrote:

Running a backup on my Mac OS X 10.11 laptop, i get an assertion error:

Gypaetus:~ markgrandi$ duplicity --encrypt-sign-key 0x2CB2C0948891C4830B11AFB6E7B785F6FA5CBE16 --log-file ~/Temp/test_duplicity.log --name "Gypaetus-mgrandi-backup" --use-agent --exclude-device-files --progress --volsize 1024 --file-prefix "Gypaetus-mgrandi-backup" --exclude-filelist ~/duplicity_exclude.txt ~/ par2+file:///Volumes/mgrandi_256SSD/tmp_backup
Reading globbing filelist /Users/markgrandi/duplicity_exclude.txt
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: none
No signatures found, switching to full backup.
Error accessing possibly locked file /Users/markgrandi/.viminfo
Error accessing possibly locked file /Users/markgrandi/Library/Saved Application State/com.adobe.flashplayer.installmanager.savedState
Reading globbing filelist /Users/markgrandi/duplicity_exclude.txt
Traceback (most recent call last):
 File "/Users/markgrandi/usr/bin/duplicity", line 1532, in <module>
   with_tempdir(main)
 File "/Users/markgrandi/usr/bin/duplicity", line 1526, in with_tempdir
   fn()
 File "/Users/markgrandi/usr/bin/duplicity", line 1380, in main
   do_backup(action)
 File "/Users/markgrandi/usr/bin/duplicity", line 1501, in do_backup
   full_backup(col_stats)
 File "/Users/markgrandi/usr/bin/duplicity", line 561, in full_backup
   sig_outfp = get_sig_fileobj("full-sig")
 File "/Users/markgrandi/usr/bin/duplicity", line 530, in get_sig_fileobj
   overwrite=True)
 File "/Users/markgrandi/usr/lib/python2.7/site-packages/duplicity/dup_temp.py", line 75, in get_fileobj_duppath
   tdp = TempDupPath(tdpname, parseresults=file_naming.parse(partname))
 File "/Users/markgrandi/usr/lib/python2.7/site-packages/duplicity/path.py", line 748, in __init__
   assert len(index) == 1
AssertionError


I don't understand this stack trace, it runs 'tdp = TempDupPath(tdpname, parseresults=file_naming.parse(partname))', but then in the super class for TempDupPath, it's hitting an AssertionError (that index is a tuple of length 1), but as you see from the caller, its explicitly leaving that parameter out, so that the default is an empty tuple, and thus fails the assertion error. However, disabling assertions doesn't work either, as it just crashes later on in the program as it tries to access index 0 of said tuple.

i'm running duplicity 0.7.06

Hello again,

Since no one commented on my post, I decided to look into why this was happening.

Running the command, as a test (only has a few files in it)

duplicity --encrypt-sign-key 0x2CB2C0948891C4830B11AFB6E7B785F6FA5CBE16 --log-file ~/Temp/test_duplicity.log --name "Gypaetus-mgrandi-backup" --use-agent --exclude-device-files --progress --volsize 1024 --file-prefix "Gypaetus" /Users/markgrandi/Temp/synctool_certs file:///Users/markgrandi/Temp/dup_back_tmp

so , the stack trace for the error is:

Traceback (most recent call last):

  File "/Users/markgrandi/usr/bin/duplicity", line 1532, in <module>

    with_tempdir(main)

  File "/Users/markgrandi/usr/bin/duplicity", line 1526, in with_tempdir

    fn()

  File "/Users/markgrandi/usr/bin/duplicity", line 1380, in main

    do_backup(action)

  File "/Users/markgrandi/usr/bin/duplicity", line 1501, in do_backup

    full_backup(col_stats)

  File "/Users/markgrandi/usr/bin/duplicity", line 561, in full_backup

    sig_outfp = get_sig_fileobj("full-sig")

  File "/Users/markgrandi/usr/bin/duplicity", line 530, in get_sig_fileobj

    overwrite=True)

  File "/Users/markgrandi/usr/lib/python2.7/site-packages/duplicity/dup_temp.py", line 75, in get_fileobj_duppath

    tdp = TempDupPath(tdpname, parseresults=file_naming.parse(partname))

  File "/Users/markgrandi/usr/lib/python2.7/site-packages/duplicity/path.py", line 749, in __init__

    assert len(index) == 1

AssertionError

so in 'duplicity/dup_temp.py:75', it tries to create a TempDupPath object. I said in my previous message that I was confused on why it didn't pass an 'index' parameter, but apparently the docstring says its OK to omit that if you pass a 'parseresults' parameter instead. So 'parseresults' is the result of 'file_naming.parse(partname)', partname is currently the string ''Gypaetusduplicity-full-signatures.20160223T040831Z.sigtar.part' . Stepping through the parse() function, the error becomes clear. The first code executed in the parse() function is 'filename = filename.lower()', but the regex it tries to run later on in parse()#check_sig(), is (with the file_prefix appended): ''^Gypaetusduplicity-full-signatures\\.(?P<time>.*?)\\.sigtar(?P<partial>(\\.part))?(\\.|$)'' . It tries to match the string ''gypaetusduplicity-full-signatures.20160223t040831z.sigtar.part'' against the regex, which fails (because of the capital letter being gone)

So the error becomes that if the file_prefix has a capital letter in it, it will fail to parse it with the regex (because the path that it passes to search() is lowercased), will return None, and then it will blow up on the 'assert len(index) == 1' in path.py:749 .

I'm not quite sure why the path is lowercased in the 'parse()' function, is it because some filesystems are case insensitive? But I can't think of a filesystem that is case insensitive that isn't case preserving as well. Either way, a quick way to fix this is to just remove the .lower() method call in parse(), but if we don't want remove the lower() call, duplicity could exit early if it sees that file_prefix has a capital letter in it?

~Mark 

_______________________________________________
Duplicity-talk mailing list
address@hidden
https://lists.nongnu.org/mailman/listinfo/duplicity-talk