Re: Tiny Guix (and containers)

[Dave Love]

Pjotr Prins <address@hidden> writes:

> For most software we just need the binary executable and its shared
> libs. That is exactly what I package with my tool:
>
> For gemma the binary closure is only 18Mb. See
> https://github.com/genetics-statistics/GEMMA/issues/90
>
> That is small. And it runs on HPC. Maybe this is the way forward for
> HPC. For many HPC tools we can create such very small tarballs.

Small size is probably mostly an issue with ramfs nodes as long as you
don't hammer a shared filesystem.  The issue is probably more the
dependencies, and what happens when you try to rebuild.  Also it can be
relevant if you want to version a whole system image or install on local
disk.

> You
> don't even need my relocatable installer if you have permission for
> /gnu/store (but then NFS would be a better idea). Great thing
> about Guix packages is that we can figure out what the dependencies
> are and strip it down to those. There is no interference. When it
> works, it works always!

But OS packages maintain dependencies, and you can supply dummy provides
to avoid sucking in too much from the base system.  There are advantages
to their typical dependencies on the basis of interfaces.

For what it's worth, you don't necessarily get what you need with Guix.
For instance, the openmpi package now doesn't depend on (a version of)
gfortran, so you need to know what to install with it to build MPI
Fortran code (or profile it).

> Once you have a Guix package and closure it is quite easy to compile
> it into something small and relocatable. That is what I have been
> experimenting with and I like the result. I am going to deploy GEMMA
> on a KNL supercomputer this month.

For interest, why specifically does the closure matter in that case?
(Something that's relevant to KNL and not in Guix is the memkind
librray.)

>> I suppose the general point is that Guix seems to have rejected
>> experience from other distributions, and it's not clear to me why.  (I
>> don't mean it should necessarily follow them, of course.)  Is there any
>> summary of the rationale for different decisions like not splitting
>> packages into development and runtime and other components, packaging
>> static libraries, and discarding debugging information, for instance?
>
> In my opinion Guix is both young and mature at the same time.
> Splitting packages is hard work and I think if enough people want to
> do it it will happen (eventually). Like Ludo suggests, we could start
> with the low hanging fruit.

Yes, on the status.  I hope that problems I see can be fixed with
engineering effort, and I've tackled some large closures to understand a
bit of what goes on.  The major effort doesn't isn't an issue in other
distributions which conventionally do split packages, though there's
often scope for reducing dependencies.  Also they normally shouldn't end
up with multiple versions of the same library, or multiple ones
supporting the same interface, though the latter is currently the case
with linear algebra in Fedora.

I mean to be constructive, and I obviously think it's worth persisting
with, but there do seem to be real issues to tackle, especially if Guix
is going to fulfil the promise for user-built software.  I hope it's
useful to raise issues.