Re: Bug#647522: non-deterministic compression results with gzip -n9

[Riku Voipio]

Thanks Cyril for tracking this down.

On Wed, Feb 08, 2012 at 01:15:00PM +0100, Cyril Brulebois wrote:
> I think at least the attached patch won't hurt (when the DYN_ALLOC part
> is fixed; and possibly turning that into a MEMSET-like macro).

> And given dh_compress is passing files in an arbitrary order (it's using
> “find” to detect files which needs to be compressed), I think we have
> an explanation about the apparently-hard-to-reproduce issues.

After some quick testing, ZEROIFY on th window is enough. However, clearing
all buffers is a good defensive strategy I think.

> Mraw,
> KiBi.

> diff --git a/gzip.c b/gzip.c
> index b867350..1153bde 100644
> --- a/gzip.c
> +++ b/gzip.c
> @@ -561,6 +561,19 @@ int main (int argc, char **argv)
>           SET_BINARY_MODE(fileno(stdout));
>       }
>          while (optind < argc) {
> +
> +         /* Make sure buffers are reset to 0 to ensure reproducibility when 
> handling several files */
> +         ZEROIFY(uch, inbuf,  INBUFSIZ +INBUF_EXTRA);
> +         ZEROIFY(uch, outbuf, OUTBUFSIZ+OUTBUF_EXTRA);
> +         ZEROIFY(ush, d_buf,  DIST_BUFSIZE);
> +         ZEROIFY(uch, window, 2L*WSIZE);
> +#ifndef MAXSEG_64K
> +         ZEROIFY(ush, tab_prefix, 1L<<BITS);
> +#else
> +         ZEROIFY(ush, tab_prefix0, 1L<<(BITS-1));
> +         ZEROIFY(ush, tab_prefix1, 1L<<(BITS-1));
> +#endif
> +
>           treat_file(argv[optind++]);
>       }
>      } else {  /* Standard input */
> diff --git a/gzip.h b/gzip.h
> index 5270c56..7a1e84b 100644
> --- a/gzip.h
> +++ b/gzip.h
> @@ -119,11 +119,13 @@ extern int method;         /* compression method */
>        array = (type*)fcalloc((size_t)(((size)+1L)/2), 2*sizeof(type)); \
>        if (!array) xalloc_die (); \
>     }
> +#  error "ZEROIFY needs an implementation, KiBi is lazy"
>  #  define FREE(array) {if (array != NULL) fcfree(array), array=NULL;}
>  #else
>  #  define EXTERN(type, array)  extern type array[]
>  #  define DECLARE(type, array, size)  type array[size]
>  #  define ALLOC(type, array, size)
> +#  define ZEROIFY(type, array, size) { for (int i=0; i<size; i++) { array[i] 
> = 0; } }
>  #  define FREE(array)
>  #endif
>