Re: Guix pull speed

[Simon Tournier]

Hi Josselin,

On Tue, 29 Aug 2023 at 13:58, Josselin Poiret <dev@jpoiret.xyz> wrote:

> After looking a bit more into guix pull speed, or to be more precise the
> "Computing Guix derivation..." step, which is not substitutable.  I've
> come to the conclusion that the thing that takes the majority of the
> time is loading the files that define the packages that the new Guix
> needs to build itself.

Well, on my machine, the bigger bottleneck seems the procedure name
’proxy’ which copies stuff around, IIUC. See [1].


>                         These files are not compiled yet, and worse,
> loading just (gnu packages guile) ends up loading 361 other package
> files.  You can generate a package graph in GraphML with `guix graph -t
> module -b graphml guile`, and use e.g. networkx to analyze it.

Hum, is this ’graphml’ something you have not submitted?  Or am I
missing a point?  Last time I played with “guix graph”, I wrote a small
script for bridging with networkx.  See [2].


> You can compare with a compiled check-out of guix by just running the
> following in a `guix repl`:
> --8<---------------cut here---------------start------------->8---
> (use-modules (guix self) (guix monad-repl))
> ,run-in-store (guix-derivation (getcwd) "0.0-git" #:pull-version 1)
> --8<---------------cut here---------------end--------------->8---
> which takes at most 5 seconds on my laptop.

Yeah, that’s fast. :-)

For comparing, what would be the corresponding derivations that “guix
pull” is building?


> One idea I had was to move all the packages that are looked up in (guix
> self) to their own little bubble, to avoid having to load extra stuff.
> However, this is not currently possible because some of them do have
> non-trivial dependency graphs.  I've identified these problematic
> inputs: guile-avahi guile-ssh guile-git guile-gnutls guix-daemon (it
> pulls in all other dependencies itself) po4a graphviz
>
> What could be done about this?  Another solution would be to somehow
> build Guix without any of the dependencies and then add them in later,
> similar to what is done with build-aux/build-self.scm to be able to load
> (guix self) in the first place.  That seems quite complex though.

Thanks for looking at this.  Well, as reported in [1], I am missing what
is the aim of the procedure ’proxy’ and if it could be avoided.

Cheers,
simon


1: Slow guix pull
Simon Tournier <zimon.toutoune@gmail.com>
Wed, 23 Aug 2023 15:48:06 +0200
id:86o7ix3m5l.fsf@gmail.com
https://lists.gnu.org/archive/html/guix-devel/2023-08
https://yhetil.org/guix/86o7ix3m5l.fsf@gmail.com

2: Some stats about the graph of dependencies
zimoun <zimon.toutoune@gmail.com>
Fri, 09 Dec 2022 18:29:43 +0100
id:874ju4qyd4.fsf@gmail.com
https://lists.gnu.org/archive/html/guix-devel/2022-12
https://yhetil.org/guix/874ju4qyd4.fsf@gmail.com