bug#45253: closed ([PATCH 0/6] Pipeline substitute integrity check, deduplication, and canonicalization)

[GNU bug Tracking System]

Your message dated Sun, 20 Dec 2020 00:04:48 +0100
with message-id <87sg818kj3.fsf@gnu.org>
and subject line Re: [bug#45253] [PATCH 0/6] Pipeline substitute integrity 
check, deduplication, and canonicalization
has caused the debbugs.gnu.org bug report #45253,
regarding [PATCH 0/6] Pipeline substitute integrity check, deduplication, and 
canonicalization
to be marked as done.

(If you believe you have received this mail in error, please contact
help-debbugs@gnu.org.)


-- 
45253: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=45253
GNU Bug Tracking System
Contact help-debbugs@gnu.org with problems
> --- Begin Message ---
>
>
>
> Subject: 
>
> [PATCH 0/6] Pipeline substitute integrity check, deduplication, and canonicalization
>
>
>
>
> Date: 
>
> Tue, 15 Dec 2020 10:38:30 +0100
>
>
>
>
> Hello Guix!
>
> This is a followup to <https://issues.guix.gnu.org/45018>.  It is
> meant to be applied on top of <https://issues.guix.gnu.org/44760>.
>
> Until now, guix-daemon would check the hash of store items just
> substituted, reset timestamps/permissions, and deduplicate.  This
> would lead to extra I/O: the whole set of files is traversed three
> times by the daemon and read two times.
>
> This patch series is about delegating that work to ‘guix substitute’,
> which it can do directly as it restores file, thereby reducing I/O
> to the minimum necessary.
>
> I tested with substitutes that contain many files:
>
>   guix build pipewire@0.2 ffmpeg ungoogled-chromium vim-full \
>     emacs-no-x emacs-no-x-toolkit
>
> On my laptop with an SSD, the wall-clock time is almost unchanged
> when fetching lzip substitutes.  You can see that the throughput
> displayed while downloading is slightly lower than before, which
> is consistent because lzip downloads are CPU-bound¹, but this is
> compensated by the lack of processing time between substitutes.
> With gzip substitutes, I see a 10% speedup on the wall-clock time
> on my laptop.
>
> Ludo’.
>
> ¹ https://lists.gnu.org/archive/html/guix-devel/2020-12/msg00177.html
>
> Ludovic Courtès (6):
>   tests: Check the build trace for hash mismatches on substitutes.
>   daemon: Let 'guix substitute' perform hash checks.
>   tests: Check the mtime and permissions of substituted items.
>   daemon: Do not reset timestamps and permissions on substituted items.
>   tests: Make sure substituted items are deduplicated.
>   daemon: Delegate deduplication to 'guix substitute'.
>
>  guix/scripts/substitute.scm | 70 +++++++++++++++++++++++++-----
>  guix/serialization.scm      |  8 +++-
>  nix/libstore/build.cc       | 85 ++++++++++++++++++++-----------------
>  tests/store.scm             | 82 +++++++++++++++++++++++++++++++++++
>  tests/substitute.scm        | 58 ++++++++++++++++++++++---
>  5 files changed, 248 insertions(+), 55 deletions(-)
>
> -- 
> 2.29.2
>
>
>
>
> --- End Message ---
> --- Begin Message ---
>
>
>
> Subject: 
>
> Re: [bug#45253] [PATCH 0/6] Pipeline substitute integrity check, deduplication, and canonicalization
>
>
>
>
> Date: 
>
> Sun, 20 Dec 2020 00:04:48 +0100
>
>
>
>
> User-agent: 
>
> Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux)
>
>
>
>
> Ludovic Courtès <ludo@gnu.org> skribis:
>
> >   tests: Check the build trace for hash mismatches on substitutes.
> >   daemon: Let 'guix substitute' perform hash checks.
> >   tests: Check the mtime and permissions of substituted items.
> >   daemon: Do not reset timestamps and permissions on substituted items.
> >   tests: Make sure substituted items are deduplicated.
> >   daemon: Delegate deduplication to 'guix substitute'.
>
> Pushed as c7c7f068c15e419aaf5ef616516aa5ad4e55c2fa.
>
> Ludo’.
>
>
> --- End Message ---