[bug#45253] [PATCH 0/6] Pipeline substitute integrity check, deduplication, and canonicalization

[Ludovic Courtès]

Ludovic Courtès <ludo@gnu.org> skribis:

> I tested with substitutes that contain many files:
>
>   guix build pipewire@0.2 ffmpeg ungoogled-chromium vim-full \
>     emacs-no-x emacs-no-x-toolkit
>
> On my laptop with an SSD, the wall-clock time is almost unchanged
> when fetching lzip substitutes.  You can see that the throughput
> displayed while downloading is slightly lower than before, which
> is consistent because lzip downloads are CPU-bound¹, but this is
> compensated by the lack of processing time between substitutes.
> With gzip substitutes, I see a 10% speedup on the wall-clock time
> on my laptop.

Picture!  First the timechart with the current daemon (gzip
substitutes, downloading from the LAN):

Notice how guix-daemon is busy in between substitute downloads: that’s
the time it takes to compute the nar hash of the store item, reset its
timestamps, and deduplicate its files.

Now the same operation after with this patch series:

This time guix-daemon remains idle all along whereas ‘guix substitute’
is almost 100% busy.  There’s no pause time between substitutes.

Ludo’.