Re: Threaded versions of cp, mv, ls for high latency / parallel filesystems?

[James Youngman]

On Sun, Nov 9, 2008 at 11:06 PM, Dr. David Alan Gilbert
<address@hidden> wrote:
> I keep wondering if the OS level needs a better interface; an 'openv' or 
> 'statv'
> or I'm currently wondering if a combined call would work - something which
> would stat a path, if it's a normal file, open it, read upto a buffers worth
> and if finished close it - it might work nicely for small files.

I suspect that a combined call would not be widely useful, though it
would likely provide a useful speedup for your use case.

I suspect that the statv/openv combination would fit more use-cases.
A statv function could be useful for anything that uses fts for
example (rm, find, ...) and for file-open dialogue boxes.

I have to say though that I've never used writev.   However, people
continue to design more advanced filesystems; the filesystem knows a
lot about how the data is arranged and (therefore) the optimal order
in which to perform operations.   The application knows a lot about
the set of operations it plans to execute, too.   However, these two
pieces of software communicate through a small keyhole; the POSIX file
API.  I'm not clear though on what nature of API might be more
generally useful for a wide class of programs; existing programs after
all are designed in ways that work well with the existing operating
system interfaces.  Perhaps this overcomplicates the issue though,
since not many programs interact with more than a few dozen files and
therefore probably wouldn't need to adopt a more complex API.

Thanks,
James.