[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Copy Times for Remote Copy

From: Phil D'Amore
Subject: Re: Copy Times for Remote Copy
Date: Fri, 30 Jan 2004 10:49:31 -0500
User-agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.3) Gecko/20030314

IIRC there was a speed enhancement fix in 2.0.6 but I don't remember there being anything significant after that. I also tried to use cfengine to solve this particular problem with much less than 27000+ files, only about 7000, and after seeing how many things cfengine did to check one file, even if it didn't have to copy it (just run with verbose and debugging sometime and watch the fun), I also came to the conclusion that cfengine isn't optimized for this sort of thing, and to make it so would probably not be trivial. I would up running with my tail between my legs back to rsync+ssh and never looked back. Since the thing I was syncing was the cfengine files repository itself, I figured that was pretty acceptable ;).

Reenen Kroukamp wrote:

On Fri, Jan 30, 2004 at 07:45:01AM -0400, Andrew Greenwood wrote:
Doesn't seem to be an issue with the network.. The SCP sends a 371MB

Try doing the copy in both directions, eg:

        scp bigfile remote_server:
        scp remote_server:bigfile .

file in 30secs.. Perhaps it is due to copying so many files??

Yes lots of files will cause delays. I concur with whoever suggested
rsync. It is definately the tool of choice for this kind of thing.

The delay _is_ quite big though. More details? Number of files,
sizes, OS, file system?


Phil D'Amore                             "Sometimes there is a fine line
Senior System Administrator               between criminally abusive
Red Hat, Inc                              behavior and fun."
Office: 919.754.3700 x44395                 -- Ted the Generic Guy
Pager: 877.383.8795                            (Dilbert 4/19/2003)

reply via email to

[Prev in Thread] Current Thread [Next in Thread]