Re: Tarlz 0.26 'make check' test failure on Debian 12

[Antonio Diaz Diaz]

Hello Aren,

Aren Tyr wrote:
> Thanks for the informative reply.

You are welcome. :-)

> So it looks like your suspicion was correct, in that it [must surely be/is] a 
> memory/RAM error (it is interesting to know that this is still a pertinent 
> hardware issue on modern machines). The machine I am running this on has 128GB 
> RAM, so quantity/resources is never an issue.

Memory errors happen even in the most expensive hardware. They almost ruined 
the mission of the Mars Science Laboratory[1]. I guess that in a machine 
with 128 GiB of RAM without ECC, a bit flip may happen almost daily[2]. 
Maybe you should test your RAM to get an idea of how reliable it is. (And it 
still could be a bug in tarlz).

[1] http://www.nongnu.org/lzip/msl-sol-200-anomaly.html
[2] https://en.wikipedia.org/wiki/ECC_memory#Research

> One minor curiosity, though I assume it is probably just to do with the 
> threading model (?): creating the archive with the -v flag so it shows the 
> files being added, stopped producing any console output after a while, until 
> the process successfully completed. This doesn't really matter, but it is nice 
> to be able to watch the files being added.

This is because the easy (single-threaded) way of printing the file names in 
the order they are added to the archive is in the grouper or in the 
muxer[1]. The grouper prints them before the workers make the real 
compression work. The muxer would print them after compression, which I 
think would be worse. Printing them in the workers would be complicated and 
the names could be printed out of order (or at the same time, as it happens 
now, if you have as many processors as files).

[1] http://www.nongnu.org/lzip/manual/tarlz_manual.html#Program-design

> from now on I'm going to always run the lzip '-tvvvv' command on the resultant 
> created archive.

Or 'lzip -tv' which is less verbose and equally effective.

> I presume I am safe in assuming that if that command completes all ok, and 
> every checksum is correct, then the archive integrity CAN be trusted :-)

If 'lzip -tv' says the archive is ok, the integrity of the archive can be 
trusted[1]. BUT 'lzip -tv' just tells you that the archive is valid, not 
that its contents match the files in the file system. If a bit flip happens 
in the input buffer before tarlz compresses it, the archive will not match 
the files. It is safer to check the archive with 'tarlz -d' just after 
creation because it compares the archive with the files in the file system.

[1] http://www.nongnu.org/lzip/safety_of_the_lzip_format.html
"After  16 years of testing, the MTBF of lzip can only be roughly estimated 
because not even one false negative has ever been observed. (And, if the 
estimate is accurate, it is not expected to ever find one in files larger 
than about 100 kB by testing)."

Best regards,
Antonio.