Re: [Duplicity-talk] duplicity use of MD4/BLAKE2 and splitting of "verify" command

[Kenneth Loafman]

New librsync was part of the plan.  The issue with the new librsync is that it doubles the size of each signature from 8 to 16.  That aggravates the problem of the non-split sigtar files since a lot of people are on the edge of the 5GB limit imposed by their storage provider.  0.8.0 will be backwards incompatible because of the increase.  We'll still be able to read old sigtar files, but older versions will not be able to read the new ones.

As to breaking verify into two commands, please make this into a bug report and I'll assign it to 0.8.

...Thanks,

...Ken

On Sun, Feb 19, 2017 at 6:28 AM, edgar.soldin--- via Duplicity-talk <address@hidden> wrote:

On 18.02.2017 23:19, Kostas Papadopoulos wrote:
> Hi duplicity devs,
>
> May I ask if it'd be possible for duplicity v0.8.x to support librsync's new (well not so new, they switched ~2 years ago) BLAKE2 format, instead of the old (and broken according to some) MD4-based checksumming ? Please see "SECURITY: MD4 collision/preimage attacks (CVE-2014-8242)" [1] [2]

agreed, raising the version number would be a good opportunity to make a clean cut.

> And while I'm asking, I'd like to request that duplicity would split current "verify" into two commands: "verify" and "compare" (note: the current syntax causes confusion and I've found this specific request repeated in many questions/posts in the duplicity-talk list since 2010). Please see output of sample test runs attached below.

this is a long standing issue. so agreed again, but that would need someone volunteering to implement it. maybe you'd like to offer a bounty or start a coding bounty campaign for this on bountysource or similar.

..ede/duply.net

> Thank you in advance for your consideration, KP
>
>
> [1] https://github.com/librsync/librsync/issues/5
>
> [2] https://bugs.launchpad.net/duplicity/+bug/1342721
>
> On 01/23/2017 10:16 PM, Kostas Papadopoulos wrote:
>> Dear duplicity devs & users,
>>
>> If the duplicity "verify" command just validates the integrity of the backup against the signatures of the sigtar files found at source_url, isn't then the requirement of a second argument (target_directory) unnecessary and confusing?
>>
>> Currently, if the specified target_dir doesn't exist, then duplicity "verify" exits with an error. If the specified target_dir is empty, it works fine. And if the specified target_dir has other random files (unrelated to duplicity), the number of "files compared" is somehow increased (e.g. from 75 to 140 in the example below)
>>
>> /root/docs was the backup original source dir
>> /tmp/t is non-existant dir
>> /tmp/tt is empty dir
>> /tmp contains other stuff too
>>
>>    address@hidden:/tmp# duplicity verify --name duply_test-local
>>     --encrypt-key AABBCCDDEEFFAA --sign-key AABBCCDDEEFFAA --verbosity
>>     4 file:///tmp/backup-test-local /root/docs
>>     Local and Remote metadata are synchronized, no sync needed.
>>     Last full backup date: Mon Jan 23 19:00:45 2017
>>     GnuPG passphrase:
>>     Verify complete: 77 files compared, 0 differences found.
>>    address@hidden:/tmp# duplicity verify --name duply_test-local
>>     --encrypt-key AABBCCDDEEFFAA --sign-key AABBCCDDEEFFAA --verbosity
>>     4 file:///tmp/backup-test-local /tmp/t
>>     Verify directory /tmp/t does not exist
>>    address@hidden:/tmp# duplicity verify --name duply_test-local
>>     --encrypt-key AABBCCDDEEFFAA --sign-key AABBCCDDEEFFAA --verbosity
>>     4 file:///tmp/backup-test-local /tmp/tt
>>     Local and Remote metadata are synchronized, no sync needed.
>>     Last full backup date: Mon Jan 23 19:00:45 2017
>>     GnuPG passphrase:
>>     Verify complete: 75 files compared, 0 differences found.
>>    address@hidden:/tmp# duplicity verify --name duply_test-local
>>     --encrypt-key AABBCCDDEEFFAA --sign-key AABBCCDDEEFFAA --verbosity
>>     4 file:///tmp/backup-test-local /tmp/
>>     Local and Remote metadata are synchronized, no sync needed.
>>     Last full backup date: Mon Jan 23 19:00:45 2017
>>     GnuPG passphrase:
>>     Verify complete: 140 files compared, 0 differences found.
>>
>> I would imagine that a more readily understandable syntax would be a duplicity "verify" command with just one argument (duplicity verify ... source_url), and another duplicity "compare" command that would take two arguments (duplicity compare ... source_url target_directory [--compare-data]) and compare backup against the target_directory's files size/date and optionally content (when used with --compare-data).
>>
>> Thank you in advance for your consideration,
>> KP
>>
>> On 04/23/2015 03:12 AM, Kenneth Loafman wrote:
>>> The signatures in the sigtar files.  Think of them as uber-large hashes.
>>>

>>> On Wed, Apr 22, 2015 at 5:25 PM, Kostas Papadopoulos <address@hidden <mailto:address@hiddengr>> wrote:
>>>
>>>     On 4/22/2015 2:51 PM, Kenneth Loafman wrote:
>>>>     Actually, he has to use --compare-data for verify to look at the
>>>>     local data, otherwise verify just validates that the backup is
>>>>     intact and is not corrupted.
>>>>
>>>
>>>     Thanks for the clarification, but then isn't the output of
>>>     "verify" a bit misleading?
>>>
>>>         ...
>>>         Last full backup date: Sun Apr  5 15:55:20 2015
>>>         Verify complete: 73 files compared, 0 differences found.
>>>         --- Finished state OK at 01:33:47.897 - Runtime 00:01:20.821 ---
>>>
>>>     Against what are those 73 files compared?
>>>
>>>     Br, KP
>>>
>>>
>>
>
>

_______________________________________________
Duplicity-talk mailing list
address@hidden
https://lists.nongnu.org/mailman/listinfo/duplicity-talk