Re: [Duplicity-talk] erroneous incrementals? verify failing? everything else seems okay?

[Kenneth Loafman]

Glad you found it.  I should have spotted it earlier.

As to sign key -- you encrypt/sign to your public keys -- then your
private keys are used for decryption and signature verification.

...Ken

Josh wrote:
> OK, looks like it was in my script... I changed some things and I'm
> getting back what I'd expect now...
> 
> backup.sh;
> -------
> #!/bin/bash
> 
> #
> # Script to backup data to Amazon S3
> #
> 
> # Export some ENV variables so you don't have to type anything
> export AWS_ACCESS_KEY_ID=<snip>
> export AWS_SECRET_ACCESS_KEY=<snip>
> export PASSPHRASE=<snip>
> 
> # The source of your backup
> SOURCE=/var/www/web2/g2data/albums/cars/jeep
> 
> # The destination
> DEST=s3+http://leveleight.test
> 
> 
> duplicity \
>     --encrypt-key=<snip> \
>     --sign-key=<snip> \
>     ${SOURCE} ${DEST}
> 
> # Reset the ENV variables. Don't need them sitting around
> export AWS_ACCESS_KEY_ID=
> export AWS_SECRET_ACCESS_KEY=
> export PASSPHRASE=
> -------
> 
> looks like it was in my SOURCE listing, then my exclude and include
> variables... I believe the script I copied was originally intended for a
> complete system backup and I'm only looking to backup a directory... So
> I just made it simpler and it now works...
> 
> last question, should I be signing my backups with my public or private key?
> 
> thanks for all your help,
> Josh
> 
> -------------
> This communication (with attachments) is covered by the U.S.C. Sec.
> 2510-252, is confidential, and intended only for the use of the
> individual or entity named above. If the reader of this message is not
> the intended recipient, you are hereby notified that any dissemination,
> distribution, or copying of this communication is strictly prohibited.
> If you receive this communication in error, notify us immediately,
> return the document(s), and destroy all copies of the original message.
> 
> 
> On Wed, Mar 25, 2009 at 7:15 AM, Josh <address@hidden
> <mailto:address@hidden>> wrote:
> 
>     okay, I just tested it with a smaller directory with only 6 photos
>     in it, thinking maybe something in my initial upload got borked
>     because it was so large or whatever... and it looks like I'm seeing
>     the same errors... so I'm thinking its gotta be my scripts... can
>     someone take a peek at them for me? here they are...
> 
>     backup.sh;
>     -------
> 
>     #!/bin/bash
> 
>     #
>     # Script to backup data to Amazon S3
>     #
> 
>     # Export some ENV variables so you don't have to type anything
>     export AWS_ACCESS_KEY_ID=<snip>
>     export AWS_SECRET_ACCESS_KEY=<snip>
>     export PASSPHRASE=<snip>
> 
>     GPG_KEY=<snip>
> 
>     # The source of your backup
>     SOURCE=/
> 
>     # The destination
>     DEST=s3+http://leveleight.test
> 
> 
>     duplicity \
>         --encrypt-key=${GPG_KEY} \
>         --sign-key=${GPG_KEY} \
>         --include=/var/www/web2/g2data/albums/cars/jeep \
> 
>         --exclude=/** \
>         ${SOURCE} ${DEST}
> 
>     # Reset the ENV variables. Don't need them sitting around
>     export AWS_ACCESS_KEY_ID=
>     export AWS_SECRET_ACCESS_KEY=
>     export PASSPHRASE=
> 
>     -------
> 
> 
>     verify.sh;
> 
>     -------
>     #!/bin/bash
> 
>     # Script to verify data on Amazon S3
> 
>     export AWS_ACCESS_KEY_ID=<snip>
>     export AWS_SECRET_ACCESS_KEY=<snip>
>     export PASSPHRASE=<snip>
>     GPG_KEY=<snip>
> 
>     duplicity verify -v9 s3+http://leveleight.test
>     /var/www/web2/g2data/albums/cars/jeep
> 
> 
>     # Reset the ENV variables. Don't need them sitting around
>     export AWS_ACCESS_KEY_ID=
>     export AWS_SECRET_ACCESS_KEY=
>     export PASSPHRASE=
>     -------
> 
> 
>     here is the output from my verify, which I did _right_ after the
>     backup (level 9 verbose);
> 
>     address@hidden:~/scripts$ ./t_amazon_verify.sh
>     Main action: verify
>     Listed
>     
> s3+http://leveleight.test/duplicity-full-signatures.20090325T120311Z.sigtar.gpg
>     Listed
>     s3+http://leveleight.test/duplicity-full.20090325T120311Z.manifest.gpg
>     Listed
>     s3+http://leveleight.test/duplicity-full.20090325T120311Z.vol1.difftar.gpg
>     Listed
>     
> s3+http://leveleight.test/duplicity-inc.20090325T120311Z.to.20090325T120322Z.manifest.gpg
>     Listed
>     
> s3+http://leveleight.test/duplicity-inc.20090325T120311Z.to.20090325T120322Z.vol1.difftar.gpg
>     Listed
>     
> s3+http://leveleight.test/duplicity-new-signatures.20090325T120311Z.to.20090325T120322Z.sigtar.gpg
>     6 files exist on backend
>     Extracting backup chains from list of files:
>     ['duplicity-full-signatures.20090325T120311Z.sigtar.gpg',
>     'duplicity-full.20090325T120311Z.manifest.gpg',
>     'duplicity-full.20090325T120311Z.vol1.difftar.gpg',
>     'duplicity-inc.20090325T120311Z.to.20090325T120322Z.manifest.gpg',
>     'duplicity-inc.20090325T120311Z.to.20090325T120322Z.vol1.difftar.gpg',
>     
> 'duplicity-new-signatures.20090325T120311Z.to.20090325T120322Z.sigtar.gpg']
>     File duplicity-full-signatures.20090325T120311Z.sigtar.gpg is not
>     part of a known set; creating new set
>     Ignoring file (rejected by backup set)
>     'duplicity-full-signatures.20090325T120311Z.sigtar.gpg'
>     File duplicity-full.20090325T120311Z.manifest.gpg is not part of a
>     known set; creating new set
>     File duplicity-full.20090325T120311Z.vol1.difftar.gpg is part of
>     known set
>     File duplicity-inc.20090325T120311Z.to.20090325T120322Z.manifest.gpg
>     is not part of a known set; creating new set
>     File
>     duplicity-inc.20090325T120311Z.to.20090325T120322Z.vol1.difftar.gpg
>     is part of known set
>     File
>     duplicity-new-signatures.20090325T120311Z.to.20090325T120322Z.sigtar.gpg
>     is not part of a known set; creating new set
>     Ignoring file (rejected by backup set)
>     'duplicity-new-signatures.20090325T120311Z.to.20090325T120322Z.sigtar.gpg'
>     Found backup chain [Wed Mar 25 07:03:11 2009]-[Wed Mar 25 07:03:11 2009]
>     Added incremental Backupset (start_time: Wed Mar 25 07:03:11 2009 /
>     end_time: Wed Mar 25 07:03:22 2009)
>     Added set Wed Mar 25 07:03:22 2009 to pre-existing chain [Wed Mar 25
>     07:03:11 2009]-[Wed Mar 25 07:03:22 2009]
> 
>     Collection Status
>     -----------------
>     Connecting with backend: BotoBackend
>     Archive dir: None
> 
>     Found 0 backup chains without signatures.
> 
>     Found a complete backup chain with matching signature chain:
>     -------------------------
>     Chain start time: Wed Mar 25 07:03:11 2009
>     Chain end time: Wed Mar 25 07:03:22 2009
> 
>     Number of contained backup sets: 2
>     Total number of contained volumes: 2
> 
>      Type of backup set:                            Time:      Num volumes:
>                     Full         Wed Mar 25 07:03:11 2009                 1
>              Incremental         Wed Mar 25 07:03:22 2009                 1
> 
>     -------------------------
>     No orphaned or incomplete backup sets found.
>     Last full backup date: Wed Mar 25 07:03:11 2009
>     Using temporary directory /tmp/duplicity-y8piFz-tempdir
>     Registering (mktemp) temporary file
>     /tmp/duplicity-y8piFz-tempdir/mktemp-Avmet--1
>     Downloading
>     s3+http://leveleight.test/duplicity-full.20090325T120311Z.manifest.gpg
>     ===== Begin GnuPG log =====
>     gpg: encrypted with 2048-bit ELG-E key, ID <snip>, created <snip>
>     "<snip> (Key for Duplicity) <address@hidden
>     <mailto:address@hidden>>"
>     gpg: Signature made <snip> using DSA key ID <snip>
>     gpg: Good signature from "<snip> (Key for Duplicity)
>     <address@hidden <mailto:address@hidden>>"
>     ===== End GnuPG log =====
>     Deleting /tmp/duplicity-y8piFz-tempdir/mktemp-Avmet--1
>     Forgetting temporary file /tmp/duplicity-y8piFz-tempdir/mktemp-Avmet--1
>     Registering (mktemp) temporary file
>     /tmp/duplicity-y8piFz-tempdir/mktemp-TCNn5_-2
>     Downloading
>     s3+http://leveleight.test/duplicity-full.20090325T120311Z.vol1.difftar.gpg
>     Registering (mktemp) temporary file
>     /tmp/duplicity-y8piFz-tempdir/mktemp-dCtSlU-3
>     Downloading
>     
> s3+http://leveleight.test/duplicity-inc.20090325T120311Z.to.20090325T120322Z.manifest.gpg
>     ===== Begin GnuPG log =====
>     gpg: encrypted with 2048-bit ELG-E key, ID <snip>, created <snip>
>     "<snip> (Key for Duplicity) <address@hidden
>     <mailto:address@hidden>>"
>     gpg: Signature made <snip> using DSA key ID <snip>
>     gpg: Good signature from "<snip> (Key for Duplicity)
>     <address@hidden <mailto:address@hidden>>"
>     ===== End GnuPG log =====
>     Deleting /tmp/duplicity-y8piFz-tempdir/mktemp-dCtSlU-3
>     Forgetting temporary file /tmp/duplicity-y8piFz-tempdir/mktemp-dCtSlU-3
>     Registering (mktemp) temporary file
>     /tmp/duplicity-y8piFz-tempdir/mktemp-WV_70M-4
>     Downloading
>     
> s3+http://leveleight.test/duplicity-inc.20090325T120311Z.to.20090325T120322Z.vol1.difftar.gpg
>     ===== Begin GnuPG log =====
>     gpg: encrypted with 2048-bit ELG-E key, ID <snip>, created <snip>
>     "<snip> (Key for Duplicity) <address@hidden
>     <mailto:address@hidden>>"
>     gpg: Signature made <snip> using DSA key ID <snip>
>     gpg: Good signature from "<snip> (Key for Duplicity)
>     <address@hidden <mailto:address@hidden>>"
>     ===== End GnuPG log =====
>     Deleting /tmp/duplicity-y8piFz-tempdir/mktemp-WV_70M-4
>     Forgetting temporary file /tmp/duplicity-y8piFz-tempdir/mktemp-WV_70M-4
>     Processed volume 1 of 2
>     Selecting /var/www/web2/g2data/albums/cars/jeep
> 
>     Difference found: File . has permissions 755, expected 755
>     Selecting /var/www/web2/g2data/albums/cars/jeep/1.jpg
>     Difference found: New file 1.jpg
>     Selecting /var/www/web2/g2data/albums/cars/jeep/2.jpg
>     Difference found: New file 2.jpg
>     Selecting /var/www/web2/g2data/albums/cars/jeep/3.jpg
>     Difference found: New file 3.jpg
>     Selecting /var/www/web2/g2data/albums/cars/jeep/4.jpg
>     Difference found: New file 4.jpg
>     Selecting /var/www/web2/g2data/albums/cars/jeep/5.jpg
>     Difference found: New file 5.jpg
>     Selecting /var/www/web2/g2data/albums/cars/jeep/6.jpg
>     Difference found: New file 6.jpg
>     Difference found: File var is missing
>     Difference found: File var/www is missing
>     Difference found: File var/www/web2 is missing
>     Difference found: File var/www/web2/g2data is missing
>     Difference found: File var/www/web2/g2data/albums is missing
>     Difference found: File var/www/web2/g2data/albums/cars is missing
>     Difference found: File var/www/web2/g2data/albums/cars/jeep is missing
>     Difference found: File var/www/web2/g2data/albums/cars/jeep/1.jpg is
>     missing
>     Difference found: File var/www/web2/g2data/albums/cars/jeep/2.jpg is
>     missing
>     Difference found: File var/www/web2/g2data/albums/cars/jeep/3.jpg is
>     missing
>     Difference found: File var/www/web2/g2data/albums/cars/jeep/4.jpg is
>     missing
>     Difference found: File var/www/web2/g2data/albums/cars/jeep/5.jpg is
>     missing
>     Difference found: File var/www/web2/g2data/albums/cars/jeep/6.jpg is
>     missing
>     ===== Begin GnuPG log =====
>     gpg: encrypted with 2048-bit ELG-E key, ID <snip>, created <snip>
>     "<snip> (Key for Duplicity) <address@hidden
>     <mailto:address@hidden>>"
>     gpg: Signature made <snip> using DSA key ID <snip>
>     gpg: Good signature from "<snip> (Key for Duplicity)
>     <address@hidden <mailto:address@hidden>>"
>     ===== End GnuPG log =====
>     Deleting /tmp/duplicity-y8piFz-tempdir/mktemp-TCNn5_-2
>     Forgetting temporary file /tmp/duplicity-y8piFz-tempdir/mktemp-TCNn5_-2
>     Processed volume 2 of 2
>     Verify complete: 20 files compared, 20 differences found.
> 
>     --end---
> 
> 
>     clearly there should be no differences, so what am I doing wrong here?
>     thanks in advance,
>     Josh
> 
> 
> 
> 
>     -------------
>     This communication (with attachments) is covered by the U.S.C. Sec.
>     2510-252, is confidential, and intended only for the use of the
>     individual or entity named above. If the reader of this message is
>     not the intended recipient, you are hereby notified that any
>     dissemination, distribution, or copying of this communication is
>     strictly prohibited. If you receive this communication in error,
>     notify us immediately, return the document(s), and destroy all
>     copies of the original message.
> 
> 
>     On Wed, Mar 25, 2009 at 6:38 AM, Edgar Soldin <address@hidden
>     <mailto:address@hidden>> wrote:
> 
> 
>         > I think the log level should be moved down to 3 (default) on
>         this one.
>         > Opinions?
>         >
> 
>         Didn't Michael kind of name the log levels or tried at least? ..
>         kind of
>         warning, info, debug ...
>         online man page states
>         -->
> 
>         *-v*/[0-9]/*, --verbosity */[0-9]/
>            Specify verbosity level (0 is total silent, 3 is the default,
>         and 9
>            is noisiest).
>         <--
> 
>         This should be more detailled. If somebody gives me an overview
>         I could
>         think of a more detailed description for the manpage.
> 
>         address@hidden .. do the --log-* commands interpret the -v
>         switch or
>         do they always put out -v9 ?
> 
>         ..ede
> 
>         > ...Ken
>         >
>         > Richard Scott wrote:
>         >
>         >> Hi,
>         >>
>         >> I have an active S3 backup routine setup and I've not had
>         issues backing up or restoring files.
>         >>
>         >> However, when I run this:
>         >>
>         >> # /usr/bin/duplicity verify -v 9
>         --archive-dir=/var/lib/duplicity/etc
>         >> s3+http://uniquefoldername/etc /etc
>         >> Verify complete: 1162 files compared, 1 difference found.
>         >>
>         >> I don't know how to display what the difference is?
>         >>
>         >> I get vary little output... I thought the "-v 9" would turn
>         on verbose output?
>         >>
>         >> I have the following installed:
>         >>
>         >> app-backup/duplicity-0.5.12
>         >> dev-lang/python-2.4.3-r4
>         >> dev-lang/python-2.5.2-r7
>         >> dev-python/boto-1.6b
>         >> dev-python/py-gnupg-0.3.2
>         >> dev-python/pycrypto-2.0.1-r6
>         >>
>         >> # python -V
>         >> Python 2.5.2
>         >>
>         >> Ken, is there any commands I can test or check for you?
>         >>
>         >> Rich.
>         >>
>         >>
>         >>> It turns out we're on the same system across the board.  I
>         thought I had
>         >>> not upgraded to 8.04.2, but it looks like I did.  32-bit
>         here as well.
>         >>>
>         >>> Not sure what's going on.  If you get the chance, try a
>         restore and make
>         >>> sure its OK.  I just don't see any problem from here.  I do
>         want to find it though, so maybe
>         >>> someone else can chip in.
>         >>>
>         >>> ...Ken
>         >>>
>         >>
>         >> _______________________________________________
>         >> Duplicity-talk mailing list
>         >> address@hidden <mailto:address@hidden>
>         >> http://lists.nongnu.org/mailman/listinfo/duplicity-talk
>         >>
>         >>
>         >
>         >
>         >
>         >
>         
> ------------------------------------------------------------------------
>         >
>         > _______________________________________________
>         > Duplicity-talk mailing list
>         > address@hidden <mailto:address@hidden>
>         > http://lists.nongnu.org/mailman/listinfo/duplicity-talk
>         >
> 
> 
> 
>         _______________________________________________
>         Duplicity-talk mailing list
>         address@hidden <mailto:address@hidden>
>         http://lists.nongnu.org/mailman/listinfo/duplicity-talk
> 
> 
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> Duplicity-talk mailing list
> address@hidden
> http://lists.nongnu.org/mailman/listinfo/duplicity-talk

signature.asc
Description: OpenPGP digital signature