Copy slower than rsync Re: Bugs and features

[Adrian Phillips]

>>>>> "Mark" == Mark Burgess <Mark.Burgess@iu.hio.no> writes:

    >> Yes, embedding the rsync algorithm as an optional way to copy
    >> files would be the only sensible way to marry cfengine & rsync.
    >> This would allow the efficiency of rsync (over slowish links
    >> with slightly changing data) with the authentication/security
    >> already present in cfengine.
    >> 
    >> If the rsync algorithm is a big win, it is a shame to have to
    >> hack up automated ssh logins and run it in a shell just to use
    >> it securely from cfengine.
    >> 

    Mark> Tests show that it isn't, except for certain types of copy
    Mark> (which probably don't occur that often on a stable
    Mark> system). The need for rsync efficiency seems to be mainly
    Mark> folklore.

To carry on this conversation from September last year :-

I have a approx. 40MB directory that is synchronised from the cfengine
server every hour. I have noticed that cfengine takes a relatively
long time to do the copy, whereas an rsync of the same area (but in
the reverse direction) takes no time at all :-

    *********************************************************************
     Main Tree Sched: copy pass 1 @ Fri Nov  8 12:29:44 2002
    *********************************************************************

    <snipped a few quick copies, takes a few seconds>

    Checking copy from cfengine:/usr/local/cfengine/dnmi/vb to /dnmi/vb
    cfengine:vbtest: Checking link from /dnmi/vb/htdocs/web/exww/index.html to 
exww.html
    Not checking whether link pointed object exists
    cfengine:vbtest: Link (/dnmi/vb/htdocs/web/exww/index.html->exww.html) 
exists.
    Checking copy from cfengine:/usr/local/cfengine/dnmi/vb/conf to 
/dnmi/vb/conf
    Saving the setuid log in /var/cfengine/cfagent..log
    Job start time set to Fri Nov  8 12:33:44 2002


    *********************************************************************
     Main Tree Sched: resolve pass 1 @ Fri Nov  8 12:33:44 2002
    *********************************************************************

Here is the copy section :-
        # Copy vb from dawn - not day copy, apache conf, or mysql databases
        # Don't use owner/group here
        ${cfrep}/dnmi/vb dest=/dnmi/vb server=${cfmaster} ignore=day 
ignore=conf ignore=databases ignore=kundedata recurse=inf

compare this to an rsync from the cfengine server to this machine :-

    freeze:~# time rsync --verbose -a -e ssh ~vb/ vbtest:/dnmi/vb 
    building file list ... done
    <snipped a few directories>
    wrote 40726 bytes  read 72 bytes  9066.22 bytes/sec
    total size is 38207548  speedup is 936.51

    real    0m4.500s
    user    0m0.110s
    sys     0m0.030s

Note, the cfengine copy takes approx. 4 minutes, whereas the similar
rsync command takes 4 seconds !!

I have the same problem with syncing my cfengine "area" from the
master cfengine server to the backup(s). Using copy would take 20 or
more minutes, using rsync takes a few seconds.

I have absolutely no idea currently the differences between cfengine's
copy code and rsync's method so don't know whether optimising copy is
the better solution or bolting rsync's library into cfengine.

Sincerely,

Adrian Phillips

-- 
Your mouse has moved.
Windows NT must be restarted for the change to take effect.
Reboot now?  [OK]