Re: [Duplicity-talk] erroneous incrementals? verify failing? everything else seems okay?

[Kenneth Loafman]

It turns out we're on the same system across the board.  I thought I had
not upgraded to 8.04.2, but it looks like I did.  32-bit here as well.

Not sure what's going on.  If you get the chance, try a restore and make
sure its OK.  I just don't see any problem from here.  I do want to find
it though, so maybe someone else can chip in.

...Ken

Josh wrote:
> I am running 32-bit
> 
> here is my backup script I used; (which when I do it a second time
> everything completes normal, without error)
> 
> #!/bin/bash
> 
> #
> # Script to backup data to Amazon S3
> #
> 
> # Export some ENV variables so you don't have to type anything
> export AWS_ACCESS_KEY_ID=<snip>
> export AWS_SECRET_ACCESS_KEY=<snip>
> export PASSPHRASE=<snip>
> 
> GPG_KEY=<snip>
> 
> # The source of your backup
> SOURCE=/
> 
> # The destination
> DEST=s3+http://leveleight.gallery
> 
> duplicity \
>     --encrypt-key=${GPG_KEY} \
>     --sign-key=${GPG_KEY} \
>     --include=/var/www/web2/g2data/albums \
>     --exclude=/** \
>     ${SOURCE} ${DEST}
> 
> # Reset the ENV variables. Don't need them sitting around
> export AWS_ACCESS_KEY_ID=
> export AWS_SECRET_ACCESS_KEY=
> export PASSPHRASE=
> 
> 
> -------------
> This communication (with attachments) is covered by the U.S.C. Sec.
> 2510-252, is confidential, and intended only for the use of the
> individual or entity named above. If the reader of this message is not
> the intended recipient, you are hereby notified that any dissemination,
> distribution, or copying of this communication is strictly prohibited.
> If you receive this communication in error, notify us immediately,
> return the document(s), and destroy all copies of the original message.
> 
> 
> On Tue, Mar 24, 2009 at 3:30 PM, Kenneth Loafman <address@hidden
> <mailto:address@hidden>> wrote:
> 
>     I just ran a 53-volume backup of .jpg files to S3 and verified.  No
>     problems found.  I'm on 8.04.1 with Boto 1.61, and duplicity 0.5.12.
>     That should be a close match to your system.
> 
>     What backup command line did you use?
> 
>     Are you 32-bit or 64-bit?
> 
>     ...Ken
> 
>     Josh wrote:
>     > Its on a Linux system.
>     >
>     > The data is mounted via the root partition (within "/" path)
>     >
>     > ext3 format
>     >
>     > -------------
>     > This communication (with attachments) is covered by the U.S.C. Sec.
>     > 2510-252, is confidential, and intended only for the use of the
>     > individual or entity named above. If the reader of this message is not
>     > the intended recipient, you are hereby notified that any
>     dissemination,
>     > distribution, or copying of this communication is strictly prohibited.
>     > If you receive this communication in error, notify us immediately,
>     > return the document(s), and destroy all copies of the original
>     message.
>     >
>     >
>     > On Tue, Mar 24, 2009 at 12:00 PM, Kenneth Loafman
>     <address@hidden <mailto:address@hidden>
>     > <mailto:address@hidden <mailto:address@hidden>>> wrote:
>     >
>     >     Don't bother finishing... I'm going to have to test some
>     against S3
>     >     myself.  I just don't see what's going on.
>     >
>     >     What about the other questions?
>     >
>     >     Is the data located on a Windows system or a Linux system?
>     >
>     >     Is it mounted, if so how?
>     >
>     >     I'm guessing its on the Linux system directly from the path,
>     but I just
>     >     want to verify that.  Do you know what kind of filesystem its on?
>     >
>     >     ...Ken
>     >
>     >     Josh wrote:
>     >     > here is the scrip I am using for the verify;
>     >     >
>     >     > -------------
>     >     > #!/bin/bash
>     >     >
>     >     > # Script to verify data on Amazon S3
>     >     >
>     >     > export AWS_ACCESS_KEY_ID=<key>
>     >     > export AWS_SECRET_ACCESS_KEY=<key>
>     >     > export PASSPHRASE=<phrase>
>     >     > GPG_KEY=<key>
>     >     >
>     >     > duplicity verify -v9 s3+http://leveleight.gallery
>     >     > /var/www/web2/g2data/albums/
>     >     >
>     >     > # Reset the ENV variables. Don't need them sitting around
>     >     > export AWS_ACCESS_KEY_ID=
>     >     > export AWS_SECRET_ACCESS_KEY=
>     >     > export PASSPHRASE=
>     >     > -------------
>     >     >
>     >     > here are some lines from when I run this script, that
>     may/not be of
>     >     > use... I still haven't let it run completely as its taking a so
>     >     long...
>     >     > but here is what I have
>     >     >
>     >     > Main action: verify
>     >     > Listed
>     >     >
>     >    
>     
> s3+http://leveleight.gallery/duplicity-full-signatures.2009-03-23T13:27:46-05:00.sigtar.gpg
>     >     > Listed
>     >     >
>     >    
>     
> s3+http://leveleight.gallery/duplicity-full.2009-03-23T13:27:46-05:00.manifest.gpg
>     >     > Listed
>     >     >
>     >    
>     
> s3+http://leveleight.gallery/duplicity-full.2009-03-23T13:27:46-05:00.vol1.difftar.gpg
>     >     > Listed
>     >     >
>     >    
>     
> s3+http://leveleight.gallery/duplicity-full.2009-03-23T13:27:46-05:00.vol10.difftar.gpg
>     >     > Listed
>     >     >
>     >    
>     
> s3+http://leveleight.gallery/duplicity-full.2009-03-23T13:27:46-05:00.vol100.difftar.gpg
>     >     > Listed
>     >     >
>     >    
>     
> s3+http://leveleight.gallery/duplicity-full.2009-03-23T13:27:46-05:00.vol1000.difftar.gpg
>     >     > Listed
>     >     >
>     >    
>     
> s3+http://leveleight.gallery/duplicity-full.2009-03-23T13:27:46-05:00.vol1001.difftar.gpg
>     >     > Listed
>     >     >
>     >    
>     
> s3+http://leveleight.gallery/duplicity-full.2009-03-23T13:27:46-05:00.vol1002.difftar.gpg
>     >     > Listed
>     >     >
>     >    
>     
> s3+http://leveleight.gallery/duplicity-full.2009-03-23T13:27:46-05:00.vol1003.difftar.gpg
>     >     > Listed
>     >     >
>     >    
>     
> s3+http://leveleight.gallery/duplicity-full.2009-03-23T13:27:46-05:00.vol1004.difftar.gpg
>     >     > Listed
>     >     >
>     >    
>     
> s3+http://leveleight.gallery/duplicity-full.2009-03-23T13:27:46-05:00.vol1005.difftar.gpg
>     >     > Listed
>     >     >
>     >    
>     
> s3+http://leveleight.gallery/duplicity-full.2009-03-23T13:27:46-05:00.vol1006.difftar.gpg
>     >     > Listed
>     >     >
>     >    
>     
> s3+http://leveleight.gallery/duplicity-full.2009-03-23T13:27:46-05:00.vol1007.difftar.gpg
>     >     > Listed
>     >     >
>     >    
>     
> s3+http://leveleight.gallery/duplicity-full.2009-03-23T13:27:46-05:00.vol1008.difftar.gpg
>     >     > Listed
>     >     >
>     >    
>     
> s3+http://leveleight.gallery/duplicity-full.2009-03-23T13:27:46-05:00.vol1009.difftar.gpg
>     >     > Listed
>     >     >
>     >    
>     
> s3+http://leveleight.gallery/duplicity-full.2009-03-23T13:27:46-05:00.vol101.difftar.gpg
>     >     > Listed
>     >     >
>     >    
>     
> s3+http://leveleight.gallery/duplicity-full.2009-03-23T13:27:46-05:00.vol1010.difftar.gpg
>     >     > Listed
>     >     >
>     >    
>     
> s3+http://leveleight.gallery/duplicity-full.2009-03-23T13:27:46-05:00.vol1011.difftar.gpg
>     >     > Listed
>     >     >
>     >    
>     
> s3+http://leveleight.gallery/duplicity-full.2009-03-23T13:27:46-05:00.vol1012.difftar.gpg
>     >     > Listed
>     >     >
>     >    
>     
> s3+http://leveleight.gallery/duplicity-full.2009-03-23T13:27:46-05:00.vol1013.difftar.gpg
>     >     > Listed
>     >     >
>     >    
>     
> s3+http://leveleight.gallery/duplicity-full.2009-03-23T13:27:46-05:00.vol1014.difftar.gpg
>     >     > Listed
>     >     >
>     >    
>     
> s3+http://leveleight.gallery/duplicity-full.2009-03-23T13:27:46-05:00.vol1015.difftar.gpg
>     >     >
>     >     > <snip>
>     >     >
>     >     > Listed
>     >     >
>     >    
>     
> s3+http://leveleight.gallery/duplicity-inc.20090323T182746Z.to.20090324T122905Z.manifest.gpg
>     >     > Listed
>     >     >
>     >    
>     
> s3+http://leveleight.gallery/duplicity-inc.20090323T182746Z.to.20090324T122905Z.vol1.difftar.gpg
>     >     > Listed
>     >     >
>     >    
>     
> s3+http://leveleight.gallery/duplicity-new-signatures.20090323T182746Z.to.20090324T122905Z.sigtar.gpg
>     >     > 1405 files exist on backend
>     >     >
>     >     > <snip>
>     >     >
>     >     > Extracting backup chains from list of files:
>     >     >
>     ['duplicity-full-signatures.2009-03-23T13:27:46-05:00.sigtar.gpg',
>     >     > 'duplicity-full.2009-03-23T13:27:46-05:00.manifest.gpg',
>     >     > 'duplicity-full.2009-03-23T13:27:46-05:00.vol1.difftar.gpg',
>     >     > 'duplicity-full.2009-03-23T13:27:46-05:00.vol10.difftar.gpg',
>     >     > 'duplicity-full.2009-03-23T13:27:46-05:00.vol100.difftar.gpg',
>     >     > 'duplicity-full.2009-03-23T13:27:46-05:00.vol1000.difftar.gpg',
>     >     > 'duplicity-full.2009-03-23T13:27:46-05:00.vol1001.difftar.gpg',
>     >     > 'duplicity-full.2009-03-23T13:27:46-05:00.vol
>     >     >
>     >     > <snip>
>     >     >
>     >     > File
>     duplicity-full-signatures.2009-03-23T13:27:46-05:00.sigtar.gpg is
>     >     > not part of a known set; creating new set
>     >     > Ignoring file (rejected by backup set)
>     >     > 'duplicity-full-signatures.2009-03-23T13:27:46-05:00.sigtar.gpg'
>     >     > File duplicity-full.2009-03-23T13:27:46-05:00.manifest.gpg
>     is not part
>     >     > of a known set; creating new set
>     >     > File
>     duplicity-full.2009-03-23T13:27:46-05:00.vol1.difftar.gpg is part
>     >     > of known set
>     >     > File
>     duplicity-full.2009-03-23T13:27:46-05:00.vol10.difftar.gpg is
>     >     part
>     >     > of known set
>     >     >
>     >     > <snip>
>     >     >
>     >     > ile
>     duplicity-full.2009-03-23T13:27:46-05:00.vol999.difftar.gpg is
>     >     part
>     >     > of known set
>     >     > File
>     >     duplicity-inc.20090323T182746Z.to.20090324T122905Z.manifest.gpg is
>     >     > not part of a known set; creating new set
>     >     > File
>     >    
>     duplicity-inc.20090323T182746Z.to.20090324T122905Z.vol1.difftar.gpg
>     >     > is part of known set
>     >     > File
>     >     >
>     >    
>     duplicity-new-signatures.20090323T182746Z.to.20090324T122905Z.sigtar.gpg
>     >     > is not part of a known set; creating new set
>     >     > Ignoring file (rejected by backup set)
>     >     >
>     >    
>     'duplicity-new-signatures.20090323T182746Z.to.20090324T122905Z.sigtar.gpg'
>     >     > Found backup chain [Mon Mar 23 13:27:46 2009]-[Mon Mar 23
>     13:27:46
>     >     2009]
>     >     > Added incremental Backupset (start_time: Mon Mar 23 13:27:46
>     2009 /
>     >     > end_time: Tue Mar 24 07:29:05 2009)
>     >     > Added set Tue Mar 24 07:29:05 2009 to pre-existing chain
>     [Mon Mar 23
>     >     > 13:27:46 2009]-[Tue Mar 24 07:29:05 2009]
>     >     > Collection Status
>     >     > -----------------
>     >     > Connecting with backend: BotoBackend
>     >     > Archive dir: None
>     >     >
>     >     > Found 0 backup chains without signatures.
>     >     >
>     >     > Found a complete backup chain with matching signature chain:
>     >     > -------------------------
>     >     > Chain start time: Mon Mar 23 13:27:46 2009
>     >     > Chain end time: Tue Mar 24 07:29:05 2009
>     >     > Number of contained backup sets: 2
>     >     > Total number of contained volumes: 1401
>     >     >  Type of backup set:                            Time:      Num
>     >     volumes:
>     >     >                 Full         Mon Mar 23 13:27:46 2009
>     >      1400
>     >     >          Incremental         Tue Mar 24 07:29:05 2009
>     >         1
>     >     > -------------------------
>     >     > No orphaned or incomplete backup sets found.
>     >     > Last full backup date: Mon Mar 23 13:27:46 2009
>     >     > Using temporary directory /tmp/duplicity-rZ55N_-tempdir
>     >     > Registering (mktemp) temporary file
>     >     > /tmp/duplicity-rZ55N_-tempdir/mktemp-jcGOJX-1
>     >     > Downloading
>     >     >
>     >    
>     
> s3+http://leveleight.gallery/duplicity-full.2009-03-23T13:27:46-05:00.manifest.gpg
>     >     > ===== Begin GnuPG log =====
>     >     > gpg: encrypted with 2048-bit ELG-E key, ID BLAHLAH, created
>     BLAHLAH
>     >     > "BLAHLAH (Key for Duplicity) <address@hidden
>     <mailto:address@hidden>
>     >     <mailto:address@hidden <mailto:address@hidden>>
>     >     > <mailto:address@hidden <mailto:address@hidden>
>     <mailto:address@hidden <mailto:address@hidden>>>>"
>     >     > gpg: Signature made BLAHLAH using DSA key ID BLAHLAH
>     >     > gpg: Good signature from "BLAHLAH (Key for Duplicity)
>     >     > <address@hidden <mailto:address@hidden>
>     <mailto:address@hidden <mailto:address@hidden>>
>     >     <mailto:address@hidden <mailto:address@hidden>
>     <mailto:address@hidden <mailto:address@hidden>>>>"
>     >     > ===== End GnuPG log =====
>     >     > Deleting /tmp/duplicity-rZ55N_-tempdir/mktemp-jcGOJX-1
>     >     > Forgetting temporary file
>     >     /tmp/duplicity-rZ55N_-tempdir/mktemp-jcGOJX-1
>     >     > Registering (mktemp) temporary file
>     >     > /tmp/duplicity-rZ55N_-tempdir/mktemp-BUaHLW-2
>     >     > Downloading
>     >     >
>     >    
>     
> s3+http://leveleight.gallery/duplicity-full.2009-03-23T13:27:46-05:00.vol1.difftar.gpg
>     >     > Registering (mktemp) temporary file
>     >     > /tmp/duplicity-rZ55N_-tempdir/mktemp-YhiFhu-3
>     >     > Downloading
>     >     >
>     >    
>     
> s3+http://leveleight.gallery/duplicity-inc.20090323T182746Z.to.20090324T122905Z.manifest.gpg
>     >     > ===== Begin GnuPG log =====
>     >     >
>     >     > <snip>
>     >     >
>     >     > Forgetting temporary file
>     >     /tmp/duplicity-rZ55N_-tempdir/mktemp-YhiFhu-3
>     >     > Registering (mktemp) temporary file
>     >     > /tmp/duplicity-rZ55N_-tempdir/mktemp-LXMvsD-4
>     >     > Downloading
>     >     >
>     >    
>     
> s3+http://leveleight.gallery/duplicity-inc.20090323T182746Z.to.20090324T122905Z.vol1.difftar.gpg
>     >     > Selecting /var/www/web2/g2data/albums
>     >     > Difference found: File . has permissions 755, expected 755
>     >     > Selecting /var/www/web2/g2data/albums/2005
>     >     > Difference found: New file 2005
>     >     > Selecting /var/www/web2/g2data/albums/2005/08-11-05_1137.jpg
>     >     > Difference found: New file 2005/08-11-05_1137.jpg
>     >     > Selecting /var/www/web2/g2data/albums/2005/08-12-05_1217.jpg
>     >     >
>     >     >
>     >     >
>     >     >
>     >     >
>     >     >
>     >     >
>     >     > -------------
>     >     > This communication (with attachments) is covered by the
>     U.S.C. Sec.
>     >     > 2510-252, is confidential, and intended only for the use of the
>     >     > individual or entity named above. If the reader of this
>     message is not
>     >     > the intended recipient, you are hereby notified that any
>     >     dissemination,
>     >     > distribution, or copying of this communication is strictly
>     prohibited.
>     >     > If you receive this communication in error, notify us
>     immediately,
>     >     > return the document(s), and destroy all copies of the original
>     >     message.
>     >     >
>     >     >
>     >     > On Tue, Mar 24, 2009 at 10:16 AM, Kenneth Loafman
>     >     <address@hidden <mailto:address@hidden>
>     <mailto:address@hidden <mailto:address@hidden>>
>     >     > <mailto:address@hidden <mailto:address@hidden>
>     <mailto:address@hidden <mailto:address@hidden>>>> wrote:
>     >     >
>     >     >     Josh wrote:
>     >     >     > I think you misunderstood, I am currently at;
>     >     >     >
>     >     >     > Ubuntu version 8.04.2
>     >     >     > duplicity version 0.5.12
>     >     >     > boto version 1.6a
>     >     >
>     >     >     Yes, I did misunderstand.  Let's start this over...
>     >     >
>     >     >     Could you show me the command lines you used?
>     >     >
>     >     >     Is the data located on a Windows system or a Linux system?
>     >     >
>     >     >     Is it mounted, if so how?
>     >     >
>     >     >     It would also help if you could run the verify with a
>     -v9 and
>     >     let us see
>     >     >     the first 200 lines and the last 200 lines of the log.  The
>     >     rest is
>     >     >     really not needed.
>     >     >
>     >     >     ...Thanks,
>     >     >     ...Ken
>     >     >
>     >     >
>     >     >     _______________________________________________
>     >     >     Duplicity-talk mailing list
>     >     >     address@hidden
>     <mailto:address@hidden> <mailto:address@hidden
>     <mailto:address@hidden>>
>     >     <mailto:address@hidden
>     <mailto:address@hidden> <mailto:address@hidden
>     <mailto:address@hidden>>>
>     >     >     http://lists.nongnu.org/mailman/listinfo/duplicity-talk
>     >     >
>     >     >
>     >     >
>     >     >
>     >    
>     ------------------------------------------------------------------------
>     >     >
>     >     > _______________________________________________
>     >     > Duplicity-talk mailing list
>     >     > address@hidden <mailto:address@hidden>
>     <mailto:address@hidden <mailto:address@hidden>>
>     >     > http://lists.nongnu.org/mailman/listinfo/duplicity-talk
>     >
>     >
>     >
>     >     _______________________________________________
>     >     Duplicity-talk mailing list
>     >     address@hidden <mailto:address@hidden>
>     <mailto:address@hidden <mailto:address@hidden>>
>     >     http://lists.nongnu.org/mailman/listinfo/duplicity-talk
>     >
>     >
>     >
>     >
>     ------------------------------------------------------------------------
>     >
>     > _______________________________________________
>     > Duplicity-talk mailing list
>     > address@hidden <mailto:address@hidden>
>     > http://lists.nongnu.org/mailman/listinfo/duplicity-talk
> 
> 
> 
>     _______________________________________________
>     Duplicity-talk mailing list
>     address@hidden <mailto:address@hidden>
>     http://lists.nongnu.org/mailman/listinfo/duplicity-talk
> 
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> Duplicity-talk mailing list
> address@hidden
> http://lists.nongnu.org/mailman/listinfo/duplicity-talk

signature.asc
Description: OpenPGP digital signature