Re: Failure of expensive test "datamash-valgrind.sh" for git HEAD

[Erik Auerswald]

Hi Tim,

On Tue, May 31, 2022 at 09:22:45PM +0000, Tim Rice wrote:
> 
> >the "expensive" test "datamash-valgrind.sh" fails on my Ubuntu 18.04
> >GNU/Linux x86-64 laptop with current git HEAD:
> >
> >```
> >$ make check-expensive TESTS=tests/datamash-valgrind.sh
> >[...]
> >custom-format failed
> >FAIL: tests/datamash-valgrind.sh
> >[...]
> >```
> 
> Yeah, I was looking at this too. I want to make sure all expensive
> tests pass before opening 1.8 up for testing. This is not the only
> expensive test that fails for me, btw. The one for doing i/o on a full
> filesystem is also a bit wonky.

I tried the filesystem image based tests only once.  They succeeded
for me.  Only the valgrind test failed.

I am not sure about the "badfs" test, because it also tests the kernel's
filesystem implementation by randomly corrupting a filesystem image.
Perhaps this could be seperated into its own test file with another
guard in addition to "expensive"?

> I thought the --format test problem could be because of something weird
> on my local machine. The test was added at the same time as the --format
> flag. I figured, surely it must have been working when it was added?

According to <https://en.wikipedia.org/wiki/Long_double> ARM64 might
use a 128-bit long double implementation that should be able to accept
all numbers in "wide".  Perhaps such a system was used for testing?

> And if datamash is intended to fail for this test, then what is the
> point of including inputs that don't cause it to crash? It would be
> clearer to go straight to the input which is too big.
> 
> Because it doesn't do that, I think datamash is intended to succeed. The
> conclusion is that the test inputs need to be pruned down.

I concur.  I'd suggest we remove input lines above a given length from
"wide" instead of hard-coding lines 9, 10, and 11 as too long, e.g.:

    awk 'length($0)<4932' wide

Thanks,
Erik