[bug#69328] [PATCH 00/12] Better source code recovery from SWH

[Ludovic Courtès]

Hello Guix!

This patch series improves source code recovery from SWH, as a followup
to <https://issues.guix.gnu.org/68741>.

It does several things:

  • ‘guix lint -c archival’ now emits save requests for VCSes other
    than Git.

  • Fix <https://issues.guix.gnu.org/69070>.

  • Allow content-addressed recovery of Mercurial and Subversion
    checkouts.

  • Allow Bazaar recovery using ‘download-nar’ (I didn’t bother with SWH).

  • Have all these things honor the ‘GUIX_DOWNLOAD_SEQUENCE’ environment
    variable.

You can try the various methods like this:

  GUIX_DOWNLOAD_SEQUENCE=nar ./pre-inst-env guix build -S apl --check
  GUIX_DOWNLOAD_SEQUENCE=swh ./pre-inst-env guix build -S guile-wisp --check
  GUIX_DOWNLOAD_SEQUENCE=swh ./pre-inst-env guix build -S guile-gcrypt --check

In the last case, note that you must be running guix-daemon for the checkout
since that uses “builtin:git-download”, which is implemented on the server
side.

There’s a few caveats:

  • Mercurial SWH fallback almost works, but not quite, due to this SWH bug:
    
<https://gitlab.softwareheritage.org/swh/infra/sysadm-environment/-/issues/5256>.

  • Right now, no Subversion checkout has the nar-sha256 ExtID at SWH for
    unclear reasons, so retrieving the source of ‘apl’ (say) from SWH
    doesn’t work yet.

  • Multi-directory Subversion downloads (‘svn-multi-fetch’) is not supported
    yet.  For that we’ll need to arrange with our SWH friends so they
    compute nar-sha256 ExtIDs for combined directories (and we’ll have to
    include that info in ‘sources.json’).

Feedback welcome!

Ludo’.

Ludovic Courtès (12):
  lint: Switch to SRFI-71.
  lint: archival: Fix crash in non-Git case.
  lint: archival: Trigger “Save Code Now” for VCSes other than Git.
  swh: Add ‘type’ field to <visit>.
  swh: ‘origin-visits’ takes an optional ‘max’ parameter.
  swh: ‘lookup-origin-revision’ handles branches pointing to
    directories.
  hg-download: Use ‘swh-download-directory-by-nar-hash’.
  svn-download: Use ‘swh-download-directory-by-nar-hash’.
  bzr-download: Implement nar fallback.
  download-nar: Distinguish ‘output’ and ‘item’ parameter.
  perform-download: Allow use of ‘download-nar’ for ‘--check’ builds.
  download: Honor ‘GUIX_DOWNLOAD_SEQUENCE’ environment variable.

 guix/build/bzr.scm                |   3 +-
 guix/build/download-nar.scm       |  12 +--
 guix/build/download.scm           |  50 +++++++---
 guix/build/git.scm                |  27 ++++--
 guix/bzr-download.scm             |  57 ++++++++---
 guix/cvs-download.scm             |  24 +++--
 guix/download.scm                 |  53 ++++-------
 guix/git-download.scm             |  20 ++--
 guix/hg-download.scm              |  36 ++++---
 guix/lint.scm                     | 151 +++++++++++++++++++-----------
 guix/scripts/perform-download.scm |  65 +++++++------
 guix/svn-download.scm             |  84 +++++++++++------
 guix/swh.scm                      |  71 ++++++++------
 tests/lint.scm                    |  20 ++++
 tests/swh.scm                     |  74 +++++++++++++++
 15 files changed, 501 insertions(+), 246 deletions(-)


base-commit: ffcce77ec488e3c89401ad77fafa65fcd9e9f5be
-- 
2.41.0