[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
bug#11950: cp: Recursively copy ordered for maximal reading speed
From: |
Michael |
Subject: |
bug#11950: cp: Recursively copy ordered for maximal reading speed |
Date: |
Mon, 16 Jul 2012 01:53:06 +0200 |
User-agent: |
claws-mail.org |
Hello,
After coding several backup tools there's something in my mind since years.
When 'cp' copies files from magnetic harddisks (commonly called after their
adapter or bus - SATA, IDE, and the like, i'm not talking about solid state)
recursively, it seems to pick up the files in 'raw' order, just as the disk
buffer spit them out (like 'in one head move'). Or so. It does not resemble any
alphabetical order, for example, it does not even stay within the same parent
folder (flingering hither and forth, as the files come in).
I suppose that's the fastest order, fastest for reading. However, one could
consider another 'maximal speed': The (later) read access of the copied files.
(Among the reasons that files are not sorted physically on disk are FS driver
gap optimizing code, and user actions like deleting single files, or moving
into another place. It could be called 'physically folder fragmentation',
something happening sooner or later, if you work on files, anyway. I'd like to
propose a way to avoid this specific fragmentation when copying.)
For example, take a large image gallery, sorted into several folders and all
files sorted alphanum. This is a standard example. Now what will file managers,
or image viewers, do with these files ? They will read in one folders content,
and display the files sorted alphanum. Usually, they even create thumbnails, so
they really access any file separately, and in the said order.
This is creating quite some disk head moves, because they are not stored in
that order 'physically' on disk. Meaning, it is slow, even if the disk is fast
and have a fast buffer, compared to the rarely existing case when the files
would be stored physically just in their access order. I hope the idea got
clear....
Now my proposal is to have a recursive 'ordered' mode, where cp copies the
files of one folder in their alphanumeric sorting (which should be the view
mode in 99% of all cases out there). It would slow down the copy process a bit,
for the benefit of later reading speed.
Now you may ask what is it good for. Aren't backups just that, and noone ever
opens them with file managers or viewers, regularly ?
But 'cp' is not only used for backups. It is also used to copy the files from
the camera chip to the harddisk in the first place, or to copy over to network
drives. I believe it is most as backend in most desktop applications anyway,
and probably in most servers too.
It still is true that most people want maximal copy speed, not maximal reading.
But maybe that's partly just because they don't know the choice even exists. If
there was such a recursive option, then backup or download tools at least could
offer it in their settings too. I would certainly use it in my backup code,
because i'm dealing with massive backups, where (maybe unobviously) speed does
not matter so much exactly for that reason: Because it needs hours anyway. I do
not need speed with backup. I need speed when reading.
I'm a DJ with huge music collection, and also a massive photographer and doing
lots of movie clips too, doing backups since more than 10 years, and i am
absolutely sure about this choice. I just think that there is a grain of
meaning in my proposal.
I'm not on any bug list, i hope this can be accepted just as a mail. Let me
know if and how i can do it better.
Kind regards, Michael
- bug#11950: cp: Recursively copy ordered for maximal reading speed,
Michael <=