Re: dd interface to posix_fadvise

[Pádraig Brady]

Paul Eggert wrote:
> Pádraig Brady <address@hidden> writes:
> 
>>> But this is what POSIX_FADV_SEQUENTIAL is designed for, no?
>>> That is, POSIX_FADV_SEQUENTIAL should be tuned for usage patterns like
>>> dd's, right?  Why isn't it?
>> There used to be an O_STREAM flag that did all this.
>> Pity they removed it in preference for the posix interface.
> 
> I don't mind the interface change, but I do mind
> POSIX_FADV_SEQUENTIAL exhibiting bad performance for straight
> sequential access.  That seems to be a clear performance bug.
> 
> It's hard to believe that the intent is that one should call
> POSIX_FADV_NOREUSE by hand instead, after each read.  That's
> enormously inconvenient.

You think that's bad. Look at the latest suggestion,
for getting the process to throw away just the data it read,
and not data that was previously cached by other processes:
http://lkml.org/lkml/2007/3/3/155
I don't agree with that, and think the kernel
should just throw away data that is referenced once.
Hopefully I'll be able to do a patch for that.

>> I'll test on debian stable also to see what the issue is.  Note
>> I noticed another issue with glibc where it always returns 0,
>> even if you do posix_fadvise() on a pipe or invalid file
>> descriptor...  The errno is set appropriately, but I don't think
>> you can generally query that if the call returns success?
> 
> According to the POSIX spec, posix_fadvise is a little unusual: it
> returns 0 on success, and an error number on failure.  errno is
> junk after calling it, I guess.
> 
> Could be that Linux got this wrong, too.

Ah thanks.
Looks like just the manpage is incorrect which confused me.
I'll send a patch.

Posix says

"Upon successful completion, posix_fadvise() shall return zero;
otherwise, an error number shall be returned to indicate the error."

Linux section 2 manpage says

"On success, zero is returned. On error, -1 is returned,
and errno is set appropriately."

>> I don't like that personally. I don't think that granularity
>> will be required in a tool like dd. I'm going with the
>> more abstract [io]flag=nocache interface for the moment,
> 
> If it's purely a performance improvement, I'd be inclined to
> enable it all the time.  That's even simpler, from the user's
> point of view. 

Well it's specific to the future data usage whether it's
better to cache or not, and dd can't know this.
I suppose one could turn it on by default if available,
and add the [io]flag=cache option?
I'd be very wary about doing that though.

> (I don't envy the job of writing the Autoconf
> macro for this one, though, as it seems to be a reallly buggy
> feature.)

We'll see. Hopefully I'll be able to sort out the debian stable prob.

thanks,
Pádraig.