Re: [Duplicity-talk] backup design question

[edgar . soldin]

Where is this thread actually going? :)

Wasn't it about using duplicity to ftp the backup vs. duplicity to local 
file system and rsync to remote? Just some more thoughts on that:

PROFTP:
I actually do ftp the backups directly but: The backup server is close 
to my servers. There are virtually no bandwidth worries. If the 
connection times out or similar duplicity so far retried & resumed the 
backups flawlessly.

PRORSYNC but also FTP:
If an uplink or whatever the connection to the repository is not stable 
or very slow the duplicity backup might fail. Because until now there is 
no resumablity it would start from the scratch again. Really a problem 
on a bigger full backup and a very slow link. So it makes sense to do 
the backup to a local file system and transfer it after it's done. This 
can of course be done with any file transfer available.
I am not sure if one must use rsync for the upload. As ftp might resume, 
this could be an option. But essentially I am using ftp with my servers 
because the backup server does _not_ something more advanced. I'd rather 
use sftp, but I can't.
Also I don't think rsync helps to gain much bandwidth with it's delta 
algorithm on an interupted upload. As the chunks usually are around 50MB 
or less .. this is also the max value one could save. This would be 
different with bigger files that had to be resumed.


and here some remarks I couldn't stop ;)

> Well the major difference for me between rsync and FTP is that rsync
> plays really well with SSH. Since FTP sends passwords in cleartext,
> using rsync with SSH you get passwords and all data encrypted. You
> have a point that it's more overhead since Duplicity files are already
> encrypted but I doubt you're going to notice the performance hit. 

I'd never compare ftp with rsync, simply because one is only a protocol 
definition and the other is a protocol and a smart application using 
this protocol. I just want to say: ftp wasn't developed to do what rsync 
does....
By using an ssh tunnel together with ftp you can also achieve encrypted 
ftp. But you would probably rather use sftp instead :)

> You
> mentioned that SSH can use symmetric keys which is a big benefit for
> me as you can setup a trust relationship between machines and then
> forgot about authentication between them. Probably the biggest
> different between the FTP and rsync protocols is that rsync can sends
> only differential changes, whereas FTP has to resend the whole file.
>   

Well sometimes the server supports resuming, but still not comparable 
because not designed that way.

> The speed at which rsync processes thousands of files for changes and
> then only sends the diffs is incredible in my opinion. Might not be a
> big deal with Duplicity since the backup files won't change on you but
> remember that you are getting complete integrity checking of the
> off-site mirror EVERYTIME you do a transfer. 

You _don't_. Except you run with the parameter -c to enable checksumming 
which makes it really slow then. All you get is the certainty that size 
and mod time of the mirror data did(n't) change.


Obviously I am in a writing mood :) .. regards ede
--
> I agree about the chroot
> jails with FTP, that is very easy to use and handy. I haven't done it
> using SSH but would be interesting in knowing how difficult that is -
> have you done it?
>
> Joel.
>
>
>   
> > Date: Sun, 15 Mar 2009 09:25:42 -0500
> > From: Brian Mathias <address@hidden>
> > Subject: Re: [Duplicity-talk] backup design question
> > To: Discussion of the backup program duplicity
> >        <address@hidden>
> > Message-ID:
> >        <address@hidden>
> > Content-Type: text/plain; charset="iso-8859-1"
> >
> > Thanks guys (including Joel), your responses were very helpful.  Any reason
> > why I should be using a protocol other than FTP for the off-site transfers?
> >
> > Here my understanding of some benefits of each:
> >
> > FTP
> > - less overhead
> > - simply way to "jail" the users' access
> >
> > SSH
> > - secured tunnel (passwords, for instance are not sent in cleartext)
> > - enhanced security to prevent brute force cracking of passwords
> > - can use keys for authentication
> >
> > What am I missing?
> >
> > -Brian
> >
> >
> > On Sun, Mar 15, 2009 at 7:39 AM, Kenneth Loafman <address@hidden>wrote:
> >
> >     
> > > jmc wrote:
> > >       
> > > > --- Brian Mathias [Sat, Mar 14, 2009 at 10:25:26AM -0500]: ---
> > > >         
> > > > > Hi All,
> > > > >
> > > > > I'm new here, so please accept my apology if this is not the appropriate
> > > > > place for this question.
> > > > >
> > > > > Originally I was looking at using rsync to keep data synced to my
> > > > >           
> > > server,
> > >       
> > > > > and then use duplicity to encrypt and send the data off-site.  My
> > > > >           
> > > question
> > >       
> > > > > If anyone has any ideas, I'd greatly appreciate your time and attention.
> > > > >           
> > > > i too am new here (and to duplicity), so i could be missing something
> > > > entirely in the nature of your question. but according to whatis(1) on
> > > > my system: duplicity (1) - Encrypted backup using rsync algorithm .
> > > >
> > > > so you get the efficiency of rsync with encryption for your backups all
> > > > for one low price. if i totally missed what you were after, i apologize
> > > > in advance.
> > > >         
> > > The suggested way of backing up using duplicity is to target backup to
> > > the local server using duplicity, then to use rsync to synchronize that
> > > backup with the remote site.  This has two advantages, one in that the
> > > local backup is available for rapid recovery if the need arises, and the
> > > remote backup is available after a site disaster.
> > >
> > > ...Ken
> > >
> > >
> > >
> > > _______________________________________________
> > > Duplicity-talk mailing list
> > > address@hidden
> > > http://lists.nongnu.org/mailman/listinfo/duplicity-talk
> > >
> > >
> > >       
> > -------------- next part --------------
> > An HTML attachment was scrubbed...
> > URL: 
> > http://lists.gnu.org/pipermail/duplicity-talk/attachments/20090315/e3b2990c/attachment.html
> >
> > ------------------------------
> >
> > _______________________________________________
> > Duplicity-talk mailing list
> > address@hidden
> > http://lists.nongnu.org/mailman/listinfo/duplicity-talk
> >
> >
> > End of Duplicity-talk Digest, Vol 68, Issue 28
> > **********************************************
> >
> >     
>
>
> _______________________________________________
> Duplicity-talk mailing list
> address@hidden
> http://lists.nongnu.org/mailman/listinfo/duplicity-talk
>