[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
bug#49439: grafts cause “guix environment” to get killed with OOM
From: |
Ludovic Courtès |
Subject: |
bug#49439: grafts cause “guix environment” to get killed with OOM |
Date: |
Thu, 29 Jul 2021 00:03:46 +0200 |
User-agent: |
Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) |
Hi,
Ludovic Courtès <ludo@gnu.org> skribis:
> Thinking about it, the grafts code depends on what’s in the store: when
> nothing is in the store, it bounces to the “build handler”, which
> accumulates the list of missing store items, until it starts building
> them.
So I can reproduce the problem Ricardo and you initially reported by
running:
./pre-inst-env guix environment pigx-scrnaseq --search-paths
after removing some of the ungrafted store items with:
guix gc -D $(guix build r-rlang --no-grafts) \
$(guix gc --references $(guix build pigx-scrnaseq --no-grafts))
The seemingly endless CPU usage and unbound memory use comes from the
‘build-accumulator’ build handler. Here, the graph of ‘pigx-scrnaseq’
has many nodes, and many of them depend on, say, ‘r-rlang’. Thus, when
‘r-rlang’ is not in the store, the grafting code keeps asking for it by
calling ‘build-derivations’, which aborts to the build handler; the
build handler saves the .drv file name and the continuation and keeps
going. But since the next package also depends on ‘r-langr’, we abort
again to the build handler, and so on.
The end result is a very long list of <unresolved> nodes, probably of
this order in this case:
$ guix graph -t reverse-package r-rlang |grep 'label = "'|wc -l
594
Presumably, the captured continuations occupy quite a bit of memory,
hence the quick growth.
I suppose one solution is to fire suspended builds when the build
handler sees a build request for a given derivation for the second time.
It needs more thought and testing…
Ludo’.
PS: Did you know ‘pigx-scrnaseq’ has twice as many nodes as
‘libreoffice’?
$ guix graph -t bag pigx-scrnaseq |grep 'label = "'|wc -l
1359
$ guix graph -t bag libreoffice |grep 'label = "'|wc -l
699
That makes it a great example to study and fix scalability issues!