Re: Efficiency of remote copy

[Mark . Burgess]

> While the rsync algorithm - as far as I understand - looks like this:
> 
> 1. Generate a request on the client and send it to the server
> 2. Process the request on the server and generate a file-list (with
>    additional data like checksums, obviously cached) and send it back to
>    the client
> 3. Process this filelist on the client to find the real differences,
>    generate the final transfer request nad transfer this request to the
>    server.
> 4. Collect the requestet files and send them back to the client (via a
>    compressed stream?)
> 
> Cfengine and scp seem to work like doing the checks on every indiviual
> file and imediately transferring it one after the other. - Couldn't 
> this be a reason for the differences and the low load on the server? 
> A "burst" transfer of the files could also eliminate the problem of 
> files to small for encryption...
> 
> What Do you think?
> 
> Regards, Daniel
> 

Yes, this is a possible reason - though it makes it hard to understand
the results measured at LISA last year. It is an obvious area where
improvements can be made in speed for small changes to large data sets,
but it is harder to understand the difference for first time copying,
unless rsync is hopping over some tests and checks. Even if the load
is more balanced, the time ought to be comparable.

It would be very interesting to get to the bottom of this.


M

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Work: +47 22453272            Email:  Mark.Burgess@iu.hio.no
Fax : +47 22453205            WWW  :  http://www.iu.hio.no/~mark
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~