Re: Copy Times for Remote Copy

[Phil D'Amore]

IIRC there was a speed enhancement fix in 2.0.6 but I don't remember 
there being anything significant after that.  I also tried to use 
cfengine to solve this particular problem with much less than 27000+ 
files, only about 7000, and after seeing how many things cfengine did to 
check one file, even if it didn't have to copy it (just run with verbose 
and debugging sometime and watch the fun), I also came to the conclusion 
that cfengine isn't optimized for this sort of thing, and to make it so 
would probably not be trivial.  I would up running with my tail between 
my legs back to rsync+ssh and never looked back.  Since the thing I was 
syncing was the cfengine files repository itself, I figured that was 
pretty acceptable ;).

Reenen Kroukamp wrote:

> On Fri, Jan 30, 2004 at 07:45:01AM -0400, Andrew Greenwood wrote:
>  
>
> > Doesn't seem to be an issue with the network.. The SCP sends a 371MB
> >    
>
> Try doing the copy in both directions, eg:
>
>         scp bigfile remote_server:
>         scp remote_server:bigfile .
>
>  
>
> > file in 30secs.. Perhaps it is due to copying so many files??
> >    
>
> Yes lots of files will cause delays. I concur with whoever suggested
> rsync. It is definately the tool of choice for this kind of thing.
>
> The delay _is_ quite big though. More details? Number of files,
> sizes, OS, file system?
>
> -Reenen
>
>  

--
Phil D'Amore                             "Sometimes there is a fine line
Senior System Administrator               between criminally abusive
Red Hat, Inc                              behavior and fun."
Office: 919.754.3700 x44395                 -- Ted the Generic Guy
Pager: 877.383.8795                            (Dilbert 4/19/2003)