Re: Treating tests as special case

[Ricardo Wurmus]

Pjotr Prins <address@hidden> writes:

> Ludo is correct that provisioning binary substitutes is one solution.
> But not cheap.  Can we guarantee keeping all substitutes? At least the
> ones with long running tests ;).

For berlin.guixsd.org we have an external storage array of a couple of
TB, which currently isn’t attached (I’ll get around to it some day).  We
can keep quite a few substitutes with that amount of space.

> Even so, with my idea of test substitutes you don't have to opt out of
> testing.  And you would still have found that bug. Those who care can
> test all they please.

I am not sure there’s an easy implementation that allows us to make
tests optional safely.  They are part of the derivation.  We could make
execution dependent on an environment variable that is set or not by the
daemon, I suppose.

> One suggestion: let's also look at tests that are *not* about
> integration or hardware/kernel configuration and allow for running them
> optionally. Stupidly running all tests that people come up with is not
> a great idea. We just run what authors decide that should be run.

We’ve already trimmed some of the longer test suites.  There are some
libraries and applications that have different test suites for different
purposes, and in those cases we picked something lighter and more
appropriate for our purposes.

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net