Re: Bug#1037264: cksum crashes intermittently with "Illegal instruction" on some Xen DomU

[Axel Beckert]

Hi, especially to Pádraig,

I wrote:
> Control: affects -1 aptitude-robot

JFYI: The fix for /etc/cron.daily/aptitude-robot, which triggers the
issue in its non-bash compatibility mode, is to change its shebang
line from "#!/bin/sh" to "#!/bin/bash". (dpkg-reconfigure dash or bash
to switch /bin/sh to bash unfortunately no more works since Debian 12.
Cc'ing andrewsh@d.o for that comment.)

Pádraig Brady wrote:
> cksum since v9.0 checks at runtime whether pclmul is supported.
> It seems that check is not working appropriately on a Xen DomU.
> The routine in question is pclmul_supported() at:
> https://github.com/coreutils/coreutils/blob/b841f111/src/cksum.c#L160-L191
> 
> That either suggests xen is incorrectly setting PCLMUL and AVX bits,
> or perhaps these two bits are not sufficient.
> Hmm I wonder do we also need to explicitly check for SSSE3 support?
> 
> I.e. I wonder does cksum built with the following help?
[…]
> diff --git a/src/cksum.c b/src/cksum.c
> index 85afab0ac..98733dadf 100644
> --- a/src/cksum.c
> +++ b/src/cksum.c
> @@ -172,7 +172,7 @@ pclmul_supported (void)
>        return false;
>      }
> 
> -  if (! (ecx & bit_PCLMUL) || ! (ecx & bit_AVX))
> +  if (! (ecx & bit_PCLMUL) || ! (ecx & bit_AVX) || ! (ecx & bit_SSSE3))
>      {
>        if (cksum_debug)
>          error (0, 0, "%s", _("pclmul support not detected"));

No, the patch unfortunately didn't help:

# dpkg -l coreutils
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name           Version        Architecture Description
+++-==============-==============-============-=================================
ii  coreutils      9.1-1+abetest1 amd64        GNU core utilities
# while :; do dd if=/dev/urandom count=1 2> /dev/null | cksum ; done
Illegal instruction
3835785655 512
1264218280 512
1265063674 512
3358845510 512
3390842004 512
658376191 512
3092360732 512
57993113 512
4257983404 512
2816803635 512
4082554882 512
1183251249 512
3097645355 512
3238771197 512
2777729543 512
3714227940 512
3331192910 512
1805379772 512
2540013463 512
294869588 512
222826476 512
1622837079 512
2515049677 512
3855944559 512
4031692020 512
4041321365 512
1802184575 512
2031964685 512
2781701490 512
460914961 512
Illegal instruction
3835252621 512
412678137 512
200496131 512
194185340 512
3286885624 512
Illegal instruction
2202092457 512
418097046 512
2216824095 512
3861063118 512
4214986749 512
259193791 512
2169514763 512
892443556 512
705097717 512
1758684834 512
2206099568 512
1780257589 512
82224867 512
Illegal instruction
2247709549 512
[…]

Thanks for trying to find a patch anyways.

> # while :; do dd if=/dev/urandom count=1 2> /dev/null | cksum ; done
> 1758277878 512
> 2101634611 512
> Illegal instruction
> Illegal instruction
> Illegal instruction
> Illegal instruction
> Illegal instruction
> Illegal instruction
> Illegal instruction
> Illegal instruction
> Illegal instruction
> Illegal instruction
> Illegal instruction
> 2704754638 512
> Illegal instruction
> 4028135672 512
> 2625667858 512
> Illegal instruction
> Illegal instruction
> Illegal instruction

One weird thing: The "Illegal instruction" happens much more seldom
today on the second affected DomU with the patched cksum (above) as
well as with the unpatched cksum (below), not sure why. Maybe this
also gives some hint on where to look for the cause of this issue.

1829747093 512
Illegal instruction
198577731 512
428043084 512
3695864207 512
2965121539 512
1048852751 512
3278958013 512
Illegal instruction
1852035202 512
2493300527 512
2163958493 512
1863124891 512
2734183826 512
1004299335 512
3257604044 512
1233477715 512
1720570219 512
3013835401 512
3175649825 512
1828643038 512
3146557230 512
911790943 512
1016865138 512
3033781151 512
Illegal instruction
2243248050 512

The DomU on which I initially discovered the issue still hits it as
hard as before or maybe even harder now:

[…]
Illegal instruction
3963278313 512
Illegal instruction
118145379 512
211261244 512
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
1435849033 512
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
Illegal instruction
[…]

This intensity happens on that DomU with both, with and without the
patch and it doesn't seem to make a difference if e.g. debug symbols
are installed or not.

                Regards, Axel
-- 
 ,''`.  |  Axel Beckert <abe@debian.org>, https://people.debian.org/~abe/
: :' :  |  Debian Developer, ftp.ch.debian.org Admin
`. `'   |  4096R: 2517 B724 C5F6 CA99 5329  6E61 2FF9 CD59 6126 16B5
  `-    |  1024D: F067 EA27 26B9 C3FC 1486  202E C09E 1D89 9593 0EDE

signature.asc
Description: PGP signature