Re: RFC: fixing the 32-bit size and time limits in gzip file format

[Paul Eggert]

On 08/16/10 22:19, Greg Roelofs wrote:
> Concatenated gzip streams made by newer gzips fail on older gzips,
> though.  I'm not sure I believe that's a worthwhile tradeoff.

I tend to agree.  We really want interoperability here.

For sizes I guess I'm willing to have gzip punt if input and output
are both streams, and to fall back on the current implementation if
that's true.  The alternative of squirreling away the compressed
stream until the input stream size is known is just too horrible to
contemplate.

If the input is a large regular file, though, we can output its size's
leading bits in the header.  And if the output is a regular file, we
can patch the input stream's size once it's known.  To make the
patching a bit more robust, we can stipulate that if the leading bit
is 1 in the new high-order size field, the actual size is unknown;
that way, if gzip is interrupted before it patches the size, the
placeholder won't be misinterpreted as a true size.