[bug#48463] gnu: Add j.

[Liliana Marie Prikler]

Am Dienstag, dem 18.01.2022 um 19:40 +0900 schrieb
elaexuotee@wilsonb.com:
> > FWIW I don't quite think that fat binaries are the issue here, but
> > that building AVX blows up the check phase in the way we currently
> > do. It's a similar issue w.r.t. check? for cross-compiling.  In my
> > opinion only the base feature set can be checked unless the CPU
> > supports the same features as the target.
> 
> Oh, good. Apparently, I mis-interpreted your original concerns.
> 
> > I think if we want to do fat binaries, the solution would be to
> > build all three variants from make-j and tests only the base
> > variants, and then have a non-substitutable wrapper package, that
> > takes that as an input and simply provides a wrapper tuned to the
> > current CPU features.
> > If you want, you can then rerun the correct tests in the wrapper
> > package.  The package returned by make-j could itself also provide
> > three binaries just in case someone doesn't want to generate the
> > wrapper package, but knows they can run ijconsole-avx2 just fine. 
> > WDYT?
> 
> Slick. Let me check my understanding against yours with some
> specifics:
> 
> - Name make-j's package 'jsoftware-j-lib' which
>   1. Builds all upstream stuff, and
>   2. Only runs libj.so (non-avx) tests;
> 
> - Create non-substitutable 'jsoftware-j' wrapper package which
>   1. Detects SSE extensions at build time and specializes the
> 'ijconsole'
>      script accordingly, and
>   2. Runs tests if and only if an avx or avx2 variant.
> 
> Does those points jibe with your thoughts on the fat binaries
> approach?
That's roughly what I was suggesting, yes.

> Glancing around at the CPU tuning stuff, it looks like tunable
> packages end up getting a 'cpu-tuning' property which holds the
> microartecture name passed to -march. AVX first landed in Sandy
> Bridge, and AVX2 in Haswell, so assuming cpu-tuning is accessible at
> build time, it should be easy enough to condition the build phase on
> those.
> 
> Regarding the check phase, here's a relevant comment in
> guix/transformations.scm(tuned-package):552:
> 
>     ;; The machine building this package may or may not be able to
> run code
>     ;; for MICRO-ARCHITECTURE.  Because of that, skip tests; they are
> run for
>     ;; the "baseline" variant anyway.
> 
> Which I read to mean that the check phase should explicitly use
> libj.so, ignoring libj-avx.so and libj-avx2.so. Running specialized
> tests in a non-substitutable wrapper seems potentially better in this
> particular case.
> 
> Thoughts?
I can't say that I agree with your reasoning here.  Essentially, what
you (or rather the J authors) are doing here is trying to outsmart the
compiler, while Guix assumes, that -march=WHATEVER ought to both
suffice in terms of performance and still be correct.  I'm leaning more
towards the assumption that Guix makes here.


If you want to enable tests on your local machine, you can still do so
by inheriting the transformed package.  Perhaps we should add a "--
with-tests" transformation as a counterpart to the already existing --
without-tests, which would forcefully enable checks that have been
disabled elsewhere.

Furthermore, tuned-package would still support substitution, you just
need to tell your CI to build for particular micro-architectures.  If
you're part of a sufficiently large dev team that uses J with Guix
tooling, that makes a difference between everyone running checks Monday
in the morning or simply fetching a build that has already succeeded on
Sunday.

Cheers