Re: [rdiff-backup-users] rdiff-backup/rsync +ssh performance comparison

[Vadim Kouzmine]

> On Mon, 6 Feb 2006, Vadim Kouzmine wrote:
> 
> > Let me summarize the information I've gathered during my tests, on two
> > platforms (Gentoo, Trustix 2.2) and different versions of python (2.2.3,
> > 2.3.5, 2.4.2):
> > 
> > - on 3GHz P4 Xeon I get ~2MB/s steady transfer through ssh, on P4 3GHz
> > it's ~1.9MB/s;
> > - I made tests ssh-ing to LOCALHOST, so no network is involved here.

Suddenly it's changed for me. Now I get about 4MB/s average on the same
hardware, same data, through the network. This is what's changed:

1) rdiff-backup 1.1.5 vs. 1.0.4 
2) python 2.4.2 vs 2.2.3
3) called as:
 "--remote-schema 'ssh -c blowfish %s /usr/local/Python/bin/rdiff-backup
--server --restrict-read-only /'"
vs
"--ssh-no-compression --restrict-read-only /"
4) openssh 4.3 vs 4.2
5) destination is external USB drive (actually slow) vs local RAID array

I don't have time to identify what exactly made it faster, but it's now
2 (!) times faster and it's great!

> 
> actually we can get rid of ssh entirely as well and just use pipes
> between the rdiff-backup and "rdiff-backup --server"... to reduce the
> variables even further.
> 
> when i compare:
> 
>       rdiff-backup src dst1
> 
> with:
> 
>       rdiff-backup --remote-schema '%s' src 'rdiff-backup --server'::dst2
> 
> i'm seeing an avg of 0.9s real-time for the first and 1.4s real-time
> for the second using an extracted copy of rdiff-backup-1.0.4.tar.gz as
> the src.  (the results are quite repeatable so i didn't bother with a
> larger src.)
> 
> my suspicion would be something serializing in the protocol...

I'll check it with netcat tunnels, and probably with netcat through ssh
forwarded port, and post results. Any other idea to make a tunnel?

Vadim