duplicity-talk
[Top][All Lists]
Advanced
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Duplicity-talk] erroneous incrementals? verify failing? everything

From: Maurizio Vitale
Subject: Re: [Duplicity-talk] erroneous incrementals? verify failing? everything else seems okay?
Date: Tue, 24 Mar 2009 23:07:50 -0400 

On Mar 24, 2009, at 6:01 PM, Kenneth Loafman wrote:
It turns out we're on the same system across the board. I thought I had 
not upgraded to 8.04.2, but it looks like I did.  32-bit here as well.
Not sure what's going on. If you get the chance, try a restore and make sure its OK. I just don't see any problem from here. I do want to find 
it though, so maybe someone else can chip in.


My recent experience is that in case of  problems w/ duplicity
and S3 is better to check the time setting for the system you use.
Other than this suggestion, I wouldn't know.

        Maurizio


...Ken

Josh wrote:

I am running 32-bit

here is my backup script I used; (which when I do it a second time
everything completes normal, without error)

#!/bin/bash

#
# Script to backup data to Amazon S3
#

# Export some ENV variables so you don't have to type anything
export AWS_ACCESS_KEY_ID=<snip>
export AWS_SECRET_ACCESS_KEY=<snip>
export PASSPHRASE=<snip>

GPG_KEY=<snip>

# The source of your backup
SOURCE=/

# The destination
DEST=s3+http://leveleight.gallery

duplicity \
    --encrypt-key=${GPG_KEY} \
    --sign-key=${GPG_KEY} \
    --include=/var/www/web2/g2data/albums \
    --exclude=/** \
    ${SOURCE} ${DEST}

# Reset the ENV variables. Don't need them sitting around
export AWS_ACCESS_KEY_ID=
export AWS_SECRET_ACCESS_KEY=
export PASSPHRASE=


-------------
This communication (with attachments) is covered by the U.S.C. Sec.
2510-252, is confidential, and intended only for the use of the
individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution, or copying of this communication is strictly prohibited. 
If you receive this communication in error, notify us immediately,
return the document(s), and destroy all copies of the original message. 


On Tue, Mar 24, 2009 at 3:30 PM, Kenneth Loafman <address@hidden
<mailto:address@hidden>> wrote:
I just ran a 53-volume backup of .jpg files to S3 and verified. No problems found. I'm on 8.04.1 with Boto 1.61, and duplicity 0.5.12. 
    That should be a close match to your system.

    What backup command line did you use?

    Are you 32-bit or 64-bit?

    ...Ken

    Josh wrote:

Its on a Linux system.

The data is mounted via the root partition (within "/" path)

ext3 format

-------------
This communication (with attachments) is covered by the U.S.C. Sec.
2510-252, is confidential, and intended only for the use of the
individual or entity named above. If the reader of this message is not 
the intended recipient, you are hereby notified that any

    dissemination,
distribution, or copying of this communication is strictly prohibited. 
If you receive this communication in error, notify us immediately,
return the document(s), and destroy all copies of the original

    message.



On Tue, Mar 24, 2009 at 12:00 PM, Kenneth Loafman

    <address@hidden <mailto:address@hidden>

<mailto:address@hidden <mailto:address@hidden>>> wrote:

    Don't bother finishing... I'm going to have to test some

    against S3

    myself.  I just don't see what's going on.

    What about the other questions?

    Is the data located on a Windows system or a Linux system?

    Is it mounted, if so how?

    I'm guessing its on the Linux system directly from the path,

    but I just
want to verify that. Do you know what kind of filesystem its on? 

    ...Ken

    Josh wrote:

here is the scrip I am using for the verify;

-------------
#!/bin/bash

# Script to verify data on Amazon S3

export AWS_ACCESS_KEY_ID=<key>
export AWS_SECRET_ACCESS_KEY=<key>
export PASSPHRASE=<phrase>
GPG_KEY=<key>

duplicity verify -v9 s3+http://leveleight.gallery
/var/www/web2/g2data/albums/

# Reset the ENV variables. Don't need them sitting around
export AWS_ACCESS_KEY_ID=
export AWS_SECRET_ACCESS_KEY=
export PASSPHRASE=
-------------

here are some lines from when I run this script, that

    may/not be of

use... I still haven't let it run completely as its taking a so

    long...

but here is what I have

Main action: verify
Listed
s3+http://leveleight.gallery/duplicity-full-signatures. 2009-03-23T13:27:46-05:00.sigtar.gpg 
Listed
s3+http://leveleight.gallery/duplicity-full. 2009-03-23T13:27:46-05:00.manifest.gpg 
Listed
s3+http://leveleight.gallery/duplicity-full. 2009-03-23T13:27:46-05:00.vol1.difftar.gpg 
Listed
s3+http://leveleight.gallery/duplicity-full. 2009-03-23T13:27:46-05:00.vol10.difftar.gpg 
Listed
s3+http://leveleight.gallery/duplicity-full. 2009-03-23T13:27:46-05:00.vol100.difftar.gpg 
Listed
s3+http://leveleight.gallery/duplicity-full. 2009-03-23T13:27:46-05:00.vol1000.difftar.gpg 
Listed
s3+http://leveleight.gallery/duplicity-full. 2009-03-23T13:27:46-05:00.vol1001.difftar.gpg 
Listed
s3+http://leveleight.gallery/duplicity-full. 2009-03-23T13:27:46-05:00.vol1002.difftar.gpg 
Listed
s3+http://leveleight.gallery/duplicity-full. 2009-03-23T13:27:46-05:00.vol1003.difftar.gpg 
Listed
s3+http://leveleight.gallery/duplicity-full. 2009-03-23T13:27:46-05:00.vol1004.difftar.gpg 
Listed
s3+http://leveleight.gallery/duplicity-full. 2009-03-23T13:27:46-05:00.vol1005.difftar.gpg 
Listed
s3+http://leveleight.gallery/duplicity-full. 2009-03-23T13:27:46-05:00.vol1006.difftar.gpg 
Listed
s3+http://leveleight.gallery/duplicity-full. 2009-03-23T13:27:46-05:00.vol1007.difftar.gpg 
Listed
s3+http://leveleight.gallery/duplicity-full. 2009-03-23T13:27:46-05:00.vol1008.difftar.gpg 
Listed
s3+http://leveleight.gallery/duplicity-full. 2009-03-23T13:27:46-05:00.vol1009.difftar.gpg 
Listed
s3+http://leveleight.gallery/duplicity-full. 2009-03-23T13:27:46-05:00.vol101.difftar.gpg 
Listed
s3+http://leveleight.gallery/duplicity-full. 2009-03-23T13:27:46-05:00.vol1010.difftar.gpg 
Listed
s3+http://leveleight.gallery/duplicity-full. 2009-03-23T13:27:46-05:00.vol1011.difftar.gpg 
Listed
s3+http://leveleight.gallery/duplicity-full. 2009-03-23T13:27:46-05:00.vol1012.difftar.gpg 
Listed
s3+http://leveleight.gallery/duplicity-full. 2009-03-23T13:27:46-05:00.vol1013.difftar.gpg 
Listed
s3+http://leveleight.gallery/duplicity-full. 2009-03-23T13:27:46-05:00.vol1014.difftar.gpg 
Listed
s3+http://leveleight.gallery/duplicity-full. 2009-03-23T13:27:46-05:00.vol1015.difftar.gpg 

<snip>

Listed
s3+http://leveleight.gallery/duplicity-inc.20090323T182746Z.to. 20090324T122905Z.manifest.gpg 
Listed
s3+http://leveleight.gallery/duplicity-inc.20090323T182746Z.to. 20090324T122905Z.vol1.difftar.gpg 
Listed
s3+http://leveleight.gallery/duplicity-new-signatures. 20090323T182746Z.to.20090324T122905Z.sigtar.gpg 
1405 files exist on backend

<snip>

Extracting backup chains from list of files:
['duplicity-full-signatures. 2009-03-23T13:27:46-05:00.sigtar.gpg', 
'duplicity-full.2009-03-23T13:27:46-05:00.manifest.gpg',
'duplicity-full.2009-03-23T13:27:46-05:00.vol1.difftar.gpg',
'duplicity-full.2009-03-23T13:27:46-05:00.vol10.difftar.gpg',
'duplicity-full.2009-03-23T13:27:46-05:00.vol100.difftar.gpg',
'duplicity-full.2009-03-23T13:27:46-05:00.vol1000.difftar.gpg',
'duplicity-full.2009-03-23T13:27:46-05:00.vol1001.difftar.gpg',
'duplicity-full.2009-03-23T13:27:46-05:00.vol

<snip>

File

    duplicity-full-signatures.2009-03-23T13:27:46-05:00.sigtar.gpg is

not part of a known set; creating new set
Ignoring file (rejected by backup set)
'duplicity-full-signatures.2009-03-23T13:27:46-05:00.sigtar.gpg'
File duplicity-full.2009-03-23T13:27:46-05:00.manifest.gpg

    is not part

of a known set; creating new set
File

    duplicity-full.2009-03-23T13:27:46-05:00.vol1.difftar.gpg is part

of known set
File

    duplicity-full.2009-03-23T13:27:46-05:00.vol10.difftar.gpg is

    part

of known set

<snip>

ile

    duplicity-full.2009-03-23T13:27:46-05:00.vol999.difftar.gpg is

    part

of known set
File
duplicity-inc.20090323T182746Z.to. 20090324T122905Z.manifest.gpg is 
not part of a known set; creating new set
File
duplicity-inc.20090323T182746Z.to. 20090324T122905Z.vol1.difftar.gpg 
is part of known set
File
duplicity-new-signatures.20090323T182746Z.to. 20090324T122905Z.sigtar.gpg 
is not part of a known set; creating new set
Ignoring file (rejected by backup set)
'duplicity-new-signatures.20090323T182746Z.to. 20090324T122905Z.sigtar.gpg' 
Found backup chain [Mon Mar 23 13:27:46 2009]-[Mon Mar 23

    13:27:46

    2009]

Added incremental Backupset (start_time: Mon Mar 23 13:27:46

    2009 /

end_time: Tue Mar 24 07:29:05 2009)
Added set Tue Mar 24 07:29:05 2009 to pre-existing chain

    [Mon Mar 23

13:27:46 2009]-[Tue Mar 24 07:29:05 2009]
Collection Status
-----------------
Connecting with backend: BotoBackend
Archive dir: None

Found 0 backup chains without signatures.

Found a complete backup chain with matching signature chain:
-------------------------
Chain start time: Mon Mar 23 13:27:46 2009
Chain end time: Tue Mar 24 07:29:05 2009
Number of contained backup sets: 2
Total number of contained volumes: 1401
 Type of backup set:                            Time:      Num

    volumes:

                Full         Mon Mar 23 13:27:46 2009

     1400

         Incremental         Tue Mar 24 07:29:05 2009

        1

-------------------------
No orphaned or incomplete backup sets found.
Last full backup date: Mon Mar 23 13:27:46 2009
Using temporary directory /tmp/duplicity-rZ55N_-tempdir
Registering (mktemp) temporary file
/tmp/duplicity-rZ55N_-tempdir/mktemp-jcGOJX-1
Downloading
s3+http://leveleight.gallery/duplicity-full. 2009-03-23T13:27:46-05:00.manifest.gpg 
===== Begin GnuPG log =====
gpg: encrypted with 2048-bit ELG-E key, ID BLAHLAH, created

    BLAHLAH

"BLAHLAH (Key for Duplicity) <address@hidden

    <mailto:address@hidden>

    <mailto:address@hidden <mailto:address@hidden>>

<mailto:address@hidden <mailto:address@hidden>

    <mailto:address@hidden <mailto:address@hidden>>>>"

gpg: Signature made BLAHLAH using DSA key ID BLAHLAH
gpg: Good signature from "BLAHLAH (Key for Duplicity)
<address@hidden <mailto:address@hidden>

    <mailto:address@hidden <mailto:address@hidden>>

    <mailto:address@hidden <mailto:address@hidden>

    <mailto:address@hidden <mailto:address@hidden>>>>"

===== End GnuPG log =====
Deleting /tmp/duplicity-rZ55N_-tempdir/mktemp-jcGOJX-1
Forgetting temporary file

    /tmp/duplicity-rZ55N_-tempdir/mktemp-jcGOJX-1

Registering (mktemp) temporary file
/tmp/duplicity-rZ55N_-tempdir/mktemp-BUaHLW-2
Downloading
s3+http://leveleight.gallery/duplicity-full. 2009-03-23T13:27:46-05:00.vol1.difftar.gpg 
Registering (mktemp) temporary file
/tmp/duplicity-rZ55N_-tempdir/mktemp-YhiFhu-3
Downloading
s3+http://leveleight.gallery/duplicity-inc.20090323T182746Z.to. 20090324T122905Z.manifest.gpg 
===== Begin GnuPG log =====

<snip>

Forgetting temporary file

    /tmp/duplicity-rZ55N_-tempdir/mktemp-YhiFhu-3

Registering (mktemp) temporary file
/tmp/duplicity-rZ55N_-tempdir/mktemp-LXMvsD-4
Downloading
s3+http://leveleight.gallery/duplicity-inc.20090323T182746Z.to. 20090324T122905Z.vol1.difftar.gpg 
Selecting /var/www/web2/g2data/albums
Difference found: File . has permissions 755, expected 755
Selecting /var/www/web2/g2data/albums/2005
Difference found: New file 2005
Selecting /var/www/web2/g2data/albums/2005/08-11-05_1137.jpg
Difference found: New file 2005/08-11-05_1137.jpg
Selecting /var/www/web2/g2data/albums/2005/08-12-05_1217.jpg







-------------
This communication (with attachments) is covered by the

    U.S.C. Sec.

2510-252, is confidential, and intended only for the use of the
individual or entity named above. If the reader of this

    message is not

the intended recipient, you are hereby notified that any

    dissemination,

distribution, or copying of this communication is strictly

    prohibited.

If you receive this communication in error, notify us

    immediately,

return the document(s), and destroy all copies of the original

    message.



On Tue, Mar 24, 2009 at 10:16 AM, Kenneth Loafman

    <address@hidden <mailto:address@hidden>

    <mailto:address@hidden <mailto:address@hidden>>

<mailto:address@hidden <mailto:address@hidden>
<mailto:address@hidden <mailto:address@hidden>>>> wrote: 

    Josh wrote:

I think you misunderstood, I am currently at;

Ubuntu version 8.04.2
duplicity version 0.5.12
boto version 1.6a


    Yes, I did misunderstand.  Let's start this over...

    Could you show me the command lines you used?

    Is the data located on a Windows system or a Linux system?

    Is it mounted, if so how?

    It would also help if you could run the verify with a

    -v9 and

    let us see

    the first 200 lines and the last 200 lines of the log.  The

    rest is

    really not needed.

    ...Thanks,
    ...Ken


    _______________________________________________
    Duplicity-talk mailing list
    address@hidden
<mailto:address@hidden> <mailto:Duplicity- address@hidden 
    <mailto:address@hidden>>

    <mailto:address@hidden
<mailto:address@hidden> <mailto:Duplicity- address@hidden 
    <mailto:address@hidden>>>

    http://lists.nongnu.org/mailman/listinfo/duplicity-talk
--------------------------------------------------------------------- --- 

_______________________________________________
Duplicity-talk mailing list
address@hidden <mailto:address@hidden>
<mailto:address@hidden <mailto:Duplicity- address@hidden>> 
http://lists.nongnu.org/mailman/listinfo/duplicity-talk




    _______________________________________________
    Duplicity-talk mailing list
    address@hidden <mailto:address@hidden>
<mailto:address@hidden <mailto:Duplicity- address@hidden>> 
    http://lists.nongnu.org/mailman/listinfo/duplicity-talk
--------------------------------------------------------------------- --- 

_______________________________________________
Duplicity-talk mailing list
address@hidden <mailto:address@hidden>
http://lists.nongnu.org/mailman/listinfo/duplicity-talk




    _______________________________________________
    Duplicity-talk mailing list
    address@hidden <mailto:address@hidden>
    http://lists.nongnu.org/mailman/listinfo/duplicity-talk
--------------------------------------------------------------------- --- 

_______________________________________________
Duplicity-talk mailing list
address@hidden
http://lists.nongnu.org/mailman/listinfo/duplicity-talk



_______________________________________________
Duplicity-talk mailing list
address@hidden
http://lists.nongnu.org/mailman/listinfo/duplicity-talk
reply via email to
[Prev in Thread] Current Thread [Next in Thread]