Re: Mirroring / distributing large amounts of data

[Daniel Riek]

On Fri, Jul 05, 2002 at 04:25:36PM +0200, Adrian Phillips wrote:
> >>>>> "Daniel" == Daniel Riek <riek@de.alcove.com> writes:
> 
>     Daniel> Hi, we are using Cfengine in a environment where we need
>     Daniel> to copy large amounts of data from one machine to
>     Daniel> another. There are mainly to scenarios: software
>     Daniel> distribution (rpm packages and tarballs) and mirroring for
>     Daniel> a failover cluster.
> 
> Large as in ? I use copy for the whole cfengine "setup" from one
> machine to a backup, approximately 1GB which takes some minutes. I can
> understand anyone trying to anything more than this having problems.
> 
Hmm. One driectory I am copying is only about 550 MB with many single
files and subdirectories. Copying takes about 18 minutes, if everything 
is copied. With no changes on the server it still takes 7 minutes with 
ctime checks. I am not quite sure if I did everything right because on
the other hand, the load on the server I am replicating from stays quite
low...

> ... 
>     Daniel> Unfortunately Cfengine seems to be very slow when doing
>     Daniel> such things.  This raises the question if anyone else
>     Daniel> tried to use Cfengine in this manner and what his
>     Daniel> experience is like?
> 
> One thing I'l like to do if possible was link in librsync and have
> some additional option to copy to make it use it instead. How much
> work this is and how much it will help I have no idea.
>
That would be very interesting, although I guess that it would be quite
some work (to much for the moment).


Regards, Daniel

-- 
Daniel Riek <riek@de.alcove.com>   -    http://www.alcove.com/de/
* Technical Manager                -    Tel.:   +49 (0)2 28 / 9 08 69 85
* ALCOVE Deutschland GmbH          -    Fax:    +49 (0)2 28 / 9 08 69 84
* Liberating Software              -    Mobil:  +49 (0)1 71 / 2 80 08 79

pgpM5LNbf0JI9.pgp
Description: PGP signature