Re: Mirroring / distributing large amounts of data

[Mark . Burgess]

In a paper (not written by me) in 2001, with the older (slower)
protocol, it was shown that cfengine was *faster* tham rsync
at distributing files first time around. Rsync is faster at
updating certain kinds of changes (that the algorithm was designed
for -- small changes to large files). If cfengine is slower
at certain things it is because it is doing extra checking
for security reasons, but that applies to secure copy. This
is also much faster since version 2, so I do not know of any
real studies on this.

In short, I just do not believe the asssertion that cfengine
is slow at copying  large amounts of filespace, compared
to rsync. It doesn't tally with experiments done. Is this
just an assumption, or have you actually tried to measure
it and compare?

I am not keen on the idea of including librsync in cfengine.
It would not be a straightforward task, and cfengine is much
more security conscious than rsync. Sometimes there is a reason
to take your time and check stuff.

M


On  5 Jul, Adrian Phillips wrote:
>>>>>> "Daniel" == Daniel Riek <riek@de.alcove.com> writes:
> 
>     Daniel> Hi, we are using Cfengine in a environment where we need
>     Daniel> to copy large amounts of data from one machine to
>     Daniel> another. There are mainly to scenarios: software
>     Daniel> distribution (rpm packages and tarballs) and mirroring for
>     Daniel> a failover cluster.
> 
> Large as in ? I use copy for the whole cfengine "setup" from one
> machine to a backup, approximately 1GB which takes some minutes. I can
> understand anyone trying to anything more than this having problems.
> 
>     Daniel> One way would be to use RSync. That is what we would do in
>     Daniel> this environment if we had no Cfengine. But as we have
>     Daniel> some security issues and rsync would require at least a
>     Daniel> minimal root acces from the mirroring machine, we would
>     Daniel> prefere to use Cfengine.
> 
>     Daniel> Another reason for using Cfengine to copy the data is the
>     Daniel> possibility to have services restarted depending on the
>     Daniel> copy...
> 
>     Daniel> Unfortunately Cfengine seems to be very slow when doing
>     Daniel> such things.  This raises the question if anyone else
>     Daniel> tried to use Cfengine in this manner and what his
>     Daniel> experience is like?
> 
> One thing I'l like to do if possible was link in librsync and have
> some additional option to copy to make it use it instead. How much
> work this is and how much it will help I have no idea.
> 
> Sincerely,
> 
> Adrian Phillips
> 



~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Work: +47 22453272            Email:  Mark.Burgess@iu.hio.no
Fax : +47 22453205            WWW  :  http://www.iu.hio.no/~mark
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~