[bug#37224] [PATCH 0/4] Add 'archival' checker for 'guix lint'

[Ludovic Courtès]

Hello Guix!

This patch series adds an ‘archival’ checker for ‘guix lint’, documented
like this:

     Checks whether the package’s source code is archived at Software
     Heritage (https://www.softwareheritage.org).

     When the source code that is not archived comes from a
     version-control system (VCS)—e.g., it’s obtained with ‘git-fetch’,
     send Software Heritage a “save” request so that it eventually
     archives it.  This ensures that the source will remain available in
     the long term, and that Guix can fall back to Software Heritage
     should the source code disappear from its original host.  The
     status of recent “save” requests can be viewed on-line
     (https://archive.softwareheritage.org/save/#requests).

     When source code is a tarball obtained with ‘url-fetch’, simply
     print a message when it is not archived.  As of this writing
     Software Heritage does not allow requests to save arbitrary
     tarballs; we are working on ways to ensure that non-VCS source code
     is also archived.

     Software Heritage limits the request rate per IP address
     (https://archive.softwareheritage.org/api/#rate-limiting).  When
     the limit is reached, ‘guix lint’ prints a message and the
     ‘archival’ checker stops doing anything until that limit has been
     reset.

Currently, only 25% of our packages are not fetched with ‘url-fetch’.
For the remaining 75%, this checker can only report whether the tarball
is missing (and apart from ftp.gnu.org and a few other exceptions, it
usually _is_ missing) and cannot actually save it.

Anyway, it’s a first step in that direction.  Feedback welcome!

The second step will be to write a “lister” for Software Heritage that
grabs the list of source code URLs from
<https://guix.gnu.org/packages.json>.  That could would run at SWH
and it could potentially grab the tarballs, not just the VCS checkouts.
Here’s are examples:

  
https://forge.softwareheritage.org/source/swh-lister/browse/master/swh/lister/packagist/lister.py
  
https://forge.softwareheritage.org/source/swh-lister/browse/master/swh/lister/gnu/lister.py

It should be quite easy for a Pythonista to write something similar
for our ‘packages.json’.  Any takers?  :-)

Ludo’.

Ludovic Courtès (4):
  tests: 'with-http-server' accepts multiple responses.
  swh: Add hooks for rate limiting handling.
  swh: Make 'commit-id?' public.
  lint: Add 'archival' checker.

 doc/guix.texi         |  25 ++++++
 guix/lint.scm         |  96 +++++++++++++++++++++-
 guix/swh.scm          |  88 ++++++++++++++++-----
 guix/tests/http.scm   |  39 +++++----
 tests/derivations.scm |  12 +--
 tests/lint.scm        | 179 ++++++++++++++++++++++++++++++++----------
 tests/swh.scm         |  41 +++++++++-
 7 files changed, 395 insertions(+), 85 deletions(-)

-- 
2.23.0