Re: [Duplicity-talk] ftp and short filenames prob

[Kenneth Loafman]

Sorry, duplicity is not your solution, for now.  It does retry on error
and will recover from most errors, but it does not have a restart
capability.  That's going to take some redesign.

Your best bet if you want to use duplicity is to partition the backup
into multiple directory trees so that if you have to restart, you only
lose the time for the portion in that tree.

Drives are cheap enough now that using a drive to stage your backup is
quite reasonable and offers advantages over slow off-site backup since
you have a local copy for quick restore.  You can then rsync that drive
to the remote site.

...Ken

AJ Weber wrote:
> Re-installing from source tarball provided significantly different
> operation (as you hinted).  I would suggest checking the RPM on the site.
>  
> So I'll continue to test this, but what's the real scoop on duplicity
> and interrupted backups (and general interrupted operation)? 
> Specifically via non-rsync targets (SCP/SFTP/FTP).
>  
> I need something that can "pick up where it left off" -- at least at a
> reasonable point.  Like if it loses the most recent 5MB archive, and
> starts by building that one again, and then continuing on...that would
> be "reasonable" to me.  If I HUP duplicity in the middle of a
> full-backup of 135GB, and it has to start all over again when I run it
> again, that's not "reasonable".
> 
> Very curious what I can do here, and what duplicity's true
> operation-cycle is.
>  
> Thanks again,
> AJ
> 
>     ----- Original Message -----
>     *From:* Kenneth Loafman <mailto:address@hidden>
>     *To:* AJ Weber <mailto:address@hidden>
>     *Sent:* Tuesday, March 11, 2008 10:33 AM
>     *Subject:* Re: [Duplicity-talk] ftp and short filenames prob
> 
>     Anything with RPM is possible.  The procedures I use should have caught
>     that, but I'll check to make sure.
> 
>     ...Ken
> 
>     AJ Weber wrote:
>     > Yeah, I've tried it both ways.  Interestingly enough, I'm
>     pretty-sure I
>     > installed from the RPM on the duplicity homepage, yet a --version
>     shows
>     > 0.4.2 (even though the package name is 0.4.9).  Is it possible
>     that RPM
>     > never got rebuilt properly?
>     > 
>     > I'll try re-installing from tarball.
>     > 
>     > -AJ
>     > 
>     >
>     >     ----- Original Message -----
>     >     *From:* Kenneth Loafman <mailto:address@hidden>
>     >     *To:* AJ Weber <mailto:address@hidden> ; Discussion of the
>     >     backup program duplicity <mailto:address@hidden>
>     >     *Sent:* Tuesday, March 11, 2008 9:27 AM
>     >     *Subject:* Re: [Duplicity-talk] ftp and short filenames prob
>     >
>     >     AJ,
>     >     
>     >     The --verify tells me you are running an older version of
>     duplicity, or
>     >     trying to run with the older commandline syntax.  The newer syntax
>     >     separates commands and options clearly, making the command the
>     first
>     >     argument and leaving off the '--'.  The newest version is
>     0.4.9 and I
>     >     would suggest upgrading to that if at all possible.  There
>     have been
>     >     many improvements over the last several months.
>     >     
>     >     Due to the way this NAS device works, it might be better to
>     use the
>     >     --short-filenames option.  This would give you Windows compatible
>     >     filenames that would not be munged by the SMB share.
>     >     
>     >     ...Ken
>     >     
>     >     AJ Weber wrote:
>     >     > The internal, target-server is a NAS device.  So I can see the
>     >     > directories via SMB (network share to a PC) or via FTP,
>     etc.  I was
>     >     > trying to use a "short cut" and view the target directory
>     from my
>     >     PC by
>     >     > simply checking the UNC path of that directory and I see crazy
>     >     filenames
>     >     > when I do that.  But if I use FTP and do a dir/ls on the
>     directory, it
>     >     > looks OK.  Must be a problem with that NAS device.
>     >     >
>     >     > Still trying to figure-out why the actual backup doesn't seem
>     >     > consistent.  I also can't get "verify" option to work properly.
>     >     And the
>     >     > man-page isn't helping me a lot.  I've tried various combos
>     of these,
>     >     > and it doesn't work (with and without directories on each url) :
>     >     >
>     >     > address@hidden backupCache]# duplicity --verify --encrypt-key
>     >     > "9C8ED9EA" --sign-key "9C8ED9EA" "ftp://address@hidden/DISK
>     >     2/duplicity"
>     >     > Command line error: Too few arguments
>     >     > See the duplicity manual page for instructions
>     >     > address@hidden backupCache]# duplicity verify --verbosity 5
>     >     > --encrypt-key "9C8ED9EA" --sign-key "9C8ED9EA" /misc/nasdisk1/LM
>     >     > "ftp://address@hidden/DISK 2/duplicity"
>     >     > Command line error: Too many arguments
>     >     > See the duplicity manual page for instructions
>     >     >
>     >     >
>     >     >     ----- Original Message -----
>     >     >     *From:* Edgar Soldin <mailto:address@hidden>
>     >     >     *To:* AJ Weber <mailto:address@hidden> ; Discussion
>     of the
>     >     >     backup program duplicity <mailto:address@hidden>
>     >     >     *Sent:* Tuesday, March 11, 2008 5:58 AM
>     >     >     *Subject:* Re: [Duplicity-talk] ftp and short filenames prob
>     >     >
>     >     >     hello peter & aj,
>     >     >
>     >     >     what looks suspicious to me is
>     >     >     Deleting
>     >     duplicity-full-signatures.2008-03-10T17:36:13-04:00.sigtar.gpg
>     >     >     from FTP server
>     >     >
>     >     >     it shouldnt do this at this point? would explain the missing
>     >     backup
>     >     >     chain?
>     >     >
>     >     >     regards ede
>     >     >
>     >     >     ps: @AJ
>     >     >      what kind of file system ist the --archive-dir ?
>     >     >      what do you mean by SMB names?
>     >     >
>     >     >     --
>     >     >
>     >     >     > OK, I found out this is partly due to the
>     target-server.  The
>     >     >     _SMB_names_ are mangled, but ftp shows the "expected names".
>     >     >     >
>     >     >     > However, unless I use the --archive-dir switch, I
>     can't list the
>     >     >     current files.  Shouldn't it be able to list the files
>     from the
>     >     >     remote server's signature file?  For example:
>     >     >     >
>     >     >     >
>     >    
>     ====================================================================
>     >     >     >
>     >     >     > # duplicity --verbosity 5 --encrypt-key "9C8ED9EA"
>     --sign-key
>     >     >     "9C8ED9EA" --archive-dir ~root/backupCache /misc/nasdisk1/LM
>     >     >     "ftp://address@hidden/DISK 2/duplicity"
>     >     >     > Listing files on FTP server
>     >     >     > No signatures found, switching to full backup.
>     >     >     > Generating delta - new file: .
>     >     >     > Generating delta - new file: Inbound Fax 33554476.pdf
>     >     >     > Generating delta - new file: incoming fax details.jpg
>     >     >     > Generating delta - new file: renditions.jpg
>     >     >     > Saving
>     duplicity-full.2008-03-10T17:36:13-04:00.vol1.difftar.gpg
>     >     >     on FTP server
>     >     >     > Saving
>     duplicity-full.2008-03-10T17:36:13-04:00.manifest.gpg on
>     >     >     FTP server
>     >     >     > Saving
>     >     >    
>     duplicity-full-signatures.2008-03-10T17:36:13-04:00.sigtar.gpg on
>     >     >     FTP server
>     >     >     > Listing files on FTP server
>     >     >     > Deleting
>     >     >    
>     duplicity-full-signatures.2008-03-10T17:36:13-04:00.sigtar.gpg
>     >     from
>     >     >     FTP server
>     >     >     > --------------[ Backup Statistics ]--------------
>     >     >     > StartTime 1205184973.73 (Mon Mar 10 17:36:13 2008)
>     >     >     > EndTime 1205184974.07 (Mon Mar 10 17:36:14 2008)
>     >     >     > ElapsedTime 0.34 (0.34 seconds)
>     >     >     > SourceFiles 4
>     >     >     > SourceFileSize 197429 (193 KB)
>     >     >     > NewFiles 4
>     >     >     > NewFileSize 197429 (193 KB)
>     >     >     > DeletedFiles 0
>     >     >     > ChangedFiles 0
>     >     >     > ChangedFileSize 0 (0 bytes)
>     >     >     > ChangedDeltaSize 0 (0 bytes)
>     >     >     > DeltaEntries 4
>     >     >     > RawDeltaSize 196917 (192 KB)
>     >     >     > TotalDestinationSizeChange 104254 (102 KB)
>     >     >     > Errors 0
>     >     >     > -------------------------------------------------
>     >     >     >
>     >     >     > /root/backupCache/tmp8sZEW3 still in tempfile list,
>     deleting
>     >     >     > /root/backupCache/tmpvBBqJf still in tempfile list,
>     deleting
>     >     >     >
>     >     >     > address@hidden backupCache]# duplicity --list
>     >     >     "ftp://address@hidden/DISK 2/duplicity"
>     >     >     > No signatures found, switching to full backup.
>     >     >     > No signature data found, unable to list files.
>     >     >     > address@hidden backupCache]# duplicity --list
>     --encrypt-key
>     >     >     "9C8ED9EA" --sign-key "9C8ED9EA" "ftp://address@hidden/DISK
>     >     >     2/duplicity"
>     >     >     > No signatures found, switching to full backup.
>     >     >     > No signature data found, unable to list files.
>     >     >     > address@hidden backupCache]# duplicity --list
>     --encrypt-key
>     >     >     "9C8ED9EA" --sign-
>     >     >     > key "9C8ED9EA"  --archive-dir ~root/backupCache
>     >     >     "ftp://address@hidden/DISK 2/duplicity"
>     >     >     > Thu Jul 19 13:04:20 2007 .
>     >     >     > Thu Jul 19 13:04:20 2007 Inbound Fax 33554476.pdf
>     >     >     > Thu Jul 19 13:00:11 2007 incoming fax details.jpg
>     >     >     > Thu Jul 19 13:03:48 2007 renditions.jpg
>     >     >     > address@hidden backupCache]#
>     >     >     >
>     >     >     >
>     >     >     > ----- Original Message -----
>     >     >     >   From: Peter Schuller
>     >     >     >   To: address@hidden
>     <mailto:address@hidden>
>     >     <mailto:address@hidden>
>     <mailto:address@hidden>
>     >     >     ; AJ Weber
>     >     >     >   Sent: Monday, March 10, 2008 4:18 PM
>     >     >     >   Subject: Re: [Duplicity-talk] ftp and short
>     filenames prob
>     >     >     >
>     >     >     >   > I can manually ftp files with "long filenames",
>     but when
>     >     >     duplicity uses it
>     >     >     >   > as a target, I get these mangled, short filenames
>     that mean
>     >     >     nothing to me.
>     >     >     >   > Worse: trying a "list-current-files" after a small
>     sync
>     >     gives
>     >     >     me "Backup
>     >     >     >   > source directory list does not exist."  So I can't
>     even
>     >     verify
>     >     >     that it's
>     >     >     >   > working properly.
>     >     >     >
>     >     >     >   This is the names of the files that make up the
>     duplicity
>     >     >     archive itself, that
>     >     >     >   is causing problems?
>     >     >     >
>     >     >     >   Could you give us an example of a mangled filename
>     versus the
>     >     >     expected name?
>     >     >     >
>     >     >     >   --
>     >     >     >   / Peter Schuller
>     >     >     >
>     >     >     >   PGP userID: 0xE9758B7D or 'Peter Schuller
>     >     >     <address@hidden>'
>     <mailto:address@hidden>'>
>     >     <mailto:address@hidden>'
>     <mailto:address@hidden>'>>
>     >     <mailto:address@hidden>'
>     <mailto:address@hidden>'>
>     >     <mailto:address@hidden>'
>     <mailto:address@hidden>'>>>
>     >     >     >   Key retrieval: Send an E-Mail to address@hidden
>     <mailto:address@hidden>
>     >     <mailto:address@hidden>
>     >     >     <mailto:address@hidden>
>     >     >     >   E-Mail: address@hidden
>     <mailto:address@hidden>
>     >     <mailto:address@hidden>
>     >     >     <mailto:address@hidden> Web:
>     http://www.scode.org
>     >     >     >
>     >     >     >
>     >     >     >
>     >     >     >
>     >     >   
>     >    
>     ------------------------------------------------------------------------
>     >     >     >
>     >     >     > _______________________________________________
>     >     >     > Duplicity-talk mailing list
>     >     >     > address@hidden
>     <mailto:address@hidden> <mailto:address@hidden>
>     >     <mailto:address@hidden>
>     >     >     > http://lists.nongnu.org/mailman/listinfo/duplicity-talk
>     >     >     >
>     >     >
>     >     >
>     >     >     --
>     >     >     public class WhoDidIt{ // A comment. I love comments
>     >     >       private static Person sender;
>     >     >
>     >     >       public static void main (String[] foo){
>     >     >
>     >     >       sender = new Person();
>     >     >       sender.setName(new String[]{"Edgar", "Soldin"});
>     >     >
>     >     >       Address address = new Address();
>     >     >       address.setStreet("Stadtweg 119");
>     >     >       address.setZip(39116);
>     >     >       address.setCity("Magdeburg");
>     >     >       address.setCountry("Germany");
>     >     >
>     >     >       sender.setAddress(address);
>     >     >
>     >     >       sender.setMobilePhone(" +49(0)171-2782880 ");
>     >     >       sender.setWebSiteUrl(" http://www.soldin.de ");
>     >     >       sender.setEmail(" address@hidden
>     <mailto:address@hidden> <mailto:address@hidden>
>     >     <mailto:address@hidden> ");
>     >     >       sender.setPGPPublicKey("
>     >     http://www.soldin.de/edgar_soldin.asc ");
>     >     >       sender.setGender(true);
>     >     >
>     >     >       System.out.println(sender.toString());
>     >     >       }
>     >     >     }
>     >     >
>     >     >
>     >     >
>     >    
>     ------------------------------------------------------------------------
>     >     >
>     >     > _______________________________________________
>     >     > Duplicity-talk mailing list
>     >     > address@hidden <mailto:address@hidden>
>     <mailto:address@hidden>
>     >     > http://lists.nongnu.org/mailman/listinfo/duplicity-talk
>     >     
>     >
>     >

signature.asc
Description: OpenPGP digital signature