Re: [Bug-wget] [PATCH 1-3/3] support HTTP compression

[Ángel González]

On 18/12/14 13:32, Yuriy M. Kaminskiy wrote:
> I preferred to save file as is, without attempting to decompress them, leaving
> decompression to user.
That isn't really user friendly.

> (Later patches only decompress files in-memory, when wget want to parse them for
> links (-r/-p options)).
>
> It also allows to resume compressed downloads (if server cached compressed
> variants and accepts Range:'d requests).
> And no unwanted decompression of .tar.gz, when server "helpfully" sets C-E: gzip
> for such files.
The bug is in the server there. (It may be useful to have a flag always 
forcing
decompression, but in the general case we should follow the spec)


> Rationale:
> a) most browsers supports only gzip and deflate; (de)compressing "bzip2" is more
> expensive, and "compress" is outdated, so I doubt a lot of servers support them.
> b) there are no support for decompressing bzip2 and compress in my patches
>
> Arguable, this list should be shortened to just "gzip":
> 1) there are confusion about format and compatibility problems with "deflate"
> [is it "raw deflate" or "zlib format"?],

It is zlib format:
> deflate
>         The "zlib" format defined in RFC 1950 [31] in combination with
>         the "deflate" compression mechanism described in RFC 1951 [29].
-- rfc2616



> > Doesn't this last patch mean that if I download
> > http://ftp.gnu.org/gnu/wget/wget-1.16.tar.gz and the compressed option
> > is enabled, then I get a plain tar, not a tgz?
> That's why I *didn't* want to make wget to save uncompressed files: some servers
> set Content-Encoding for already compressed files.
The server may not always provide compressed (eg. compressing static 
files but not cgi output)