Re: Treating tests as special case

[Ricardo Wurmus]

Pjotr Prins <address@hidden> writes:

> If all the inputs are the same the test will *always* pass. There is
> no point to it! The only way such a test won't pass it by divine
> intervention or real hardware problems. Both we don't want to test
> for.
>
> If tests are so important to rerun: tell me why we are not running
> tests when substituting binaries?

I don’t understand this.  People only run tests when they haven’t been
run on the build farm, because that’s part of the build.  So when the
tests have passed (and the few short phases after that), then we have
substitutes anyway, and so users won’t re-run tests.

If you get substitutes you don’t need to run the tests.

Any change here seems to only affect the case where you build locally
even though there are substitutes.  I’d say that this is a pretty rare
use case.  Build farms do this, but they build binaries (and if they
differ from binaries built elsewhere the tests may also behave
differently).

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net