Re: [sysvinit-devel] Switch release tarballs to xz

[Matias Fonzo]

On Sat, 5 May 2018 10:29:51 -0300
Jesse Smith <address@hidden> wrote:

> > Hi Jesse,
> >
> > I request to as the maintainer of sysvinit to reject this change.
> > Please do not do that, xz is not well-designed.  See:
> >
> > http://lzip.nongnu.org/xz_inadequate.html  
> 
> I looked over the linked document and the discussion Guillem mentioned
> on the Debian mailing list. I have a few thoughts on this debate.
> 
> 1. The linked document seems to be as politically motivated as it is
> technically motivated. And, having been on the receiving end of this
> propaganda effort at another project where similar claims were
> debunked, I am disinclined to believe the author's conclusions.
> 
> 2. Guillem is right, the concerns raised in the Debian bug reports and
> mailing lists were shown to be baseless.
> 
> 3. The xz compression software is used by many projects, including
> several Linux distributions which means it is used to compress a lot
> of packages

We can't think in it as a something "standardized" because it is not.

Even upstream distributions like Slackware have had problems with xz
related to the threads[1] handling:

"
--threads <number>  For xz/plzip compressed packages, set the max
                    number of threads to be used for compression. Only
has an effect on large packages. For plzip, the default is
                    equal to the number of CPU threads available on the
                    machine. For xz, the default is equal to 2 (due to
                    commonly occuring memory related failures when using
                    many threads with multi-threaded xz compression).
"

[1]
http://slackware.osuosl.org/slackware64-current/source/a/pkgtools/scripts/makepkg

> probably well over a million archives. If the document
> linked above were accurate we could expect there to be thousands of
> examples of unrecoverable archives, even if corruption was a problem
> less than 0.1% of the time. This does not appear to be the case. Even
> if it were, we can always create new archives from the git tree.

None of the adopters have studied xz in deep before its adoption. They
simply added it, like in the Linux Kernel case; you could read their
mailing lists, and you will know that what I am saying here is true.

> 4. I tested xz against our current bzip2 compression and xz does
> produce smaller archives, making it a more appealing option.

If this is the case, Lzip can compress more than xz:
http://lzip.nongnu.org/lzip_benchmark.html

Also, we can't argue that xz is better than bzip2 just because it
compress more.  This is not the only aspect of a compressor, it just a
practical aspect, but not the only.  Some projects migrated from bzip2
to xz (like GCC), but they *never* give a valid argument why xz is
better than bzip2 for switching (besides of the compression level, of
course).  One cool thing of bzip2 is that it offer a tool to recover
its files, far as I know xz does not have such tool...

> Given the above points, I am inclined to switch to xz, unless there
> is a neutral party that has evidence that using xz results in data
> loss more often than should be expected from plain file system
> corruption?

I am CC'ing this email to Antonio Diaz Diaz (author of lzip) and also
including Maria Bisen to have a different opinion coming from the
Debian project.