Re: dd interface to posix_fadvise

[Pádraig Brady]

Paul Eggert wrote:
> Pádraig Brady <address@hidden> writes:
> 
>> I was thinking it would be useful to
>> add an option to dd to tell it to
>> instruct the OS not to cache the data.
> 
> Yes, that's handy, but doesn't "dd iflag=direct" already do that for you?

Not exactly. That bypasses all the kernelly goodness.
For example if the file is already cached you needlessly read from disk.
Also readahead for performance is also not done.

> Hmm, I see that Linus doesn't like O_DIRECT
> <http://kerneltrap.org/node/7563>.  Perhaps we should also support the
> method that Linus prefers.  Can you fill us in as to why that is?

I didn't know about linus' views on O_DIRECT.
They match mine though, so they're definitely right :)

>> Note, posix_fadvise(POSIX_FADV_DONTNEED) on linux,
>> invalidates the cache for a file rather than a process....
>> This is not what is usually required, and
>> I'm working on the kernel guys to get them
>> to change this.
> 
> Yes, thanks, that sounds right.

FYI, I'm thinking one can achieve this
by just invalidating pages that have a reference count of 1.

> But dd always accesses the file sequentially.  Wouldn't
> POSIX_FADV_SEQUENTIAL be better?  or perhaps POSIX_FADV_NOREUSE?  And
> if not, why not?

They might be good to set, but probably not:
http://lxr.linux.no/source/mm/fadvise.c
http://www.die.net/doc/linux/man/man2/posix_fadvise.2.html

> I just tried the following naive patch on my Debian stable host and dd
> immediately dumped core inside posix_fadvise.  This is not a good
> sign.  It sounds like, despite Linus's wishes, posix_fadvise is not
> really ready for prime_time.


> diff --git a/src/dd.c b/src/dd.c
> +  posix_fadvise (STDIN_FILENO, 0, 0, POSIX_FADV_SEQUENTIAL);
> +  posix_fadvise (STDOUT_FILENO, 0, 0, POSIX_FADV_SEQUENTIAL);

That's mad. That trival patch works fine here on
coreutils-6.2 + glibc-2.3.5-10 + 2.6.16-1.2111_FC4 at least.

Pádraig.