Re: RFC: fixing the 32-bit size and time limits in gzip file format

[Mark Adler]

On Aug 16, 2010, at 4:16 PM, Paul Eggert wrote:
> We really want interoperability here.

However it does not seem possible to both have interoperability and to always 
provide a correct uncompressed length in the stream.

Since we'd have to give up on interoperability to completely solve the problem, 
then the obvious solution is to use one of the reserved bits (bit 6 or 7 in 
flags) to signal a variable size extra field after the current trailer.  
(Variable in that the length of the extra field is not provided in the gzip 
header, but rather in the extra field itself.)  Then we have an extensible 
means to add data to the end of a gzip stream, including the uncompressed 
length (which itself could be variable in length, since the gzip format may 
persist past the year 2040 when we may find that 63 bits is not enough).

Current versions of gzip and pigz would correctly reject the stream since a 
reserved bit is set.  (I just checked.)  That has the upside to strongly 
encourage updating to new versions of the tools.

Mark