Re: feature request: gzip/bzip support for sort

[Jim Meyering]

Dan Hipschman <address@hidden> wrote:
> On Sat, Jan 13, 2007 at 10:07:59PM -0800, Paul Eggert wrote:
>> 3.  I can see where the user might be able to specify a better
>> algorithm, for a particular data set.  For that, how about if we have
>> a --compress-program=PROGRAM option, which lets the user plug in any
>> program that works as a pipeline?  E.g., --compress-program=gzip would
>> use gzip.  The default would be to use "PROGRAM -d" to decompress; we
>> could have another option if that doesn't suffice.
>
> This is pretty close.  There is just a few more problems that I see:

One more thing to consider:

    In your proposed patch, a failed fork evokes an immediate exit.
    In that case, it may be worthwhile to retry, or even to revert
    to using gzip directly.

Consider sort's merge phase, and imagine having 40 compressed files,
each hooked to a process running gzip -d.  It's not hard to imagine
an environment in which some of those fork calls would fail.