bug#69596: ‘guix publish’ memory leak

[Ludovic Courtès]

Hi!

Ludovic Courtès <ludo@gnu.org> skribis:

> It seems that ‘guix publish’ has been leaking memory noticeably I’d say
> since the beginning of the year on berlin.  After roughly 10 days, it
> has several GiB resident and needs to be restarted or it becomes too
> unresponsive.
>
> I wonder what could be causing this because we used to let it run for
> months without problems I believe.
>
> This is what’s currently running on berlin:
>
>   
> /gnu/store/cpnshv80n3mar5lwp4qqa2dxxxv4zb03-guix-1.4.0-16.aeb4943/libexec/guix/guile
>  \
>   /gnu/store/cpnshv80n3mar5lwp4qqa2dxxxv4zb03-guix-1.4.0-16.aeb4943/bin/guix \
>   publish -u guix-publish -p 3000 -C lzip:9 -C zstd:19 --nar-path=nar \
>   --listen=localhost --workers=8 --ttl=15552000s \
>   --cache=/var/cache/guix/publish --cache-bypass-threshold=157286400
>
> … coming from commit 21e4d6cd6913eca131f2c0fd0cd509fc843c7eb8.

It turned out to be a guile-lzlib leak that had always been present:

  
https://notabug.org/guile-lzlib/guile-lzlib/commit/74bd35b690801a10ed775d486fffc7372b1b341c

The reason we were seeing it more on berlin is probably because we
increased the cache-bypass-threshold, which goes through the
‘make-lzip-output-port’ code path (as opposed to
‘call-with-lzip-output-port’).

The bug could be reproduced with:

  guix publish -p 8124 … &
  while true ; do wget -q -O/dev/full 
http://localhost:8124/nar/lzip/…-coreutils-9.1 ; done

(Replace the ellipses with the actual store file name of coreutils.)

Fixed by commit 7cef6b7ba555a9dfaf6d09cb7e112b0df77d5141, which updates
guile-lzlib.

Ludo’.