[bug#33899] [PATCH 0/5] Distributing substitutes over IPFS

[Ludovic Courtès]

Hello Guix!

Here is a first draft adding support to distribute and retrieve substitutes
over IPFS.  This builds on discussions at the R-B Summit with Héctor Sanjuan
of IPFS, lewo of Nix, Pierre Neidhardt, and also on the work Florian
Paul Schmidt posted on guix-devel last month.

The IPFS daemon exposes an HTTP API and the (guix ipfs) module provides
bindings to a subset of that API.  This module also implements a custom
“directory” format to store directory trees in IPFS (IPFS already provides
“UnixFS” and “tar” but they store too many or too few file attributes.)

‘guix publish’ and ‘guix substitute’ use (guix ipfs) to
store and retrieve store items.  Complete directory trees are stored in
IPFS “as is”, rather than as compressed archives (nars).  This allows for
deduplication in IPFS.  ‘guix publish’ adds a new “IPFS” field in
narinfos and ‘guix substitute’ can then query those objects over IPFS.
So the idea is that you still get narinfos over HTTP(S), and then you
have the option of downloading substitutes over IPFS.

I’ve pushed these patches in ‘wip-ipfs-substitutes’.  This is rough on the
edges and probably buggy, but the adventurous among us might want to give
it a spin.  :-)

Thanks,
Ludo’.

Ludovic Courtès (5):
  Add (guix json).
  tests: 'file=?' now recurses on directories.
  Add (guix ipfs).
  publish: Add IPFS support.
  DRAFT substitute: Add IPFS support.

 Makefile.am                 |   3 +
 doc/guix.texi               |  33 +++++
 guix/ipfs.scm               | 250 ++++++++++++++++++++++++++++++++++++
 guix/json.scm               |  63 +++++++++
 guix/scripts/publish.scm    |  67 +++++++---
 guix/scripts/substitute.scm | 106 ++++++++-------
 guix/swh.scm                |  35 +----
 guix/tests.scm              |  26 +++-
 tests/ipfs.scm              |  55 ++++++++
 9 files changed, 535 insertions(+), 103 deletions(-)
 create mode 100644 guix/ipfs.scm
 create mode 100644 guix/json.scm
 create mode 100644 tests/ipfs.scm

-- 
2.20.1