47/66: programming-2022: Add README.md for artifact evaluation.

[Ludovic Courtès]

civodul pushed a commit to branch master
in repository maintenance.

commit 82cf10ae6e0f4a9f4b7849b68b852567a180a48b
Author: Ludovic Courtès <ludo@gnu.org>
AuthorDate: Thu Apr 7 12:23:31 2022 +0200

    programming-2022: Add README.md for artifact evaluation.
    
    As per <https://programming-journal.org/artifact-evaluation/>.
    
    * doc/programming-2022/artifact/README.md: New file.
---
 doc/programming-2022/artifact/README.md | 283 ++++++++++++++++++++++++++++++++
 1 file changed, 283 insertions(+)

diff --git a/doc/programming-2022/artifact/README.md 
b/doc/programming-2022/artifact/README.md
new file mode 100644
index 0000000..1c81dea
--- /dev/null
+++ b/doc/programming-2022/artifact/README.md
@@ -0,0 +1,283 @@
+# Building a Secure Software Supply Chain with GNU Guix
+
+*Artifact Evaluation*
+
+# Getting Started
+
+The artifact is a virtual machine (VM) image, in QCOW2 format, which can
+be used by most emulators:
+
+  - image:
+    https://ftp.gnu.org/gnu/guix/guix-system-vm-image-1.3.0.x86_64-linux.qcow2
+  - OpenPGP signature:
+    
https://ftp.gnu.org/gnu/guix/guix-system-vm-image-1.3.0.x86_64-linux.qcow2.sig
+  - SHA256: 980312e9c94204537a9e0f7fca7d765fc316686a8bbaff111989688ce353b16c
+  - usage instructions:
+    https://guix.gnu.org/manual/en/html_node/Running-Guix-in-a-VM.html
+
+In a nutshell, you can run the following commands on an x86_64 GNU/Linux
+system to retrieve and authenticate the image:
+
+```
+wget -O /tmp/qemu-image \
+  https://ftp.gnu.org/gnu/guix/guix-system-vm-image-1.3.0.x86_64-linux.qcow2
+wget -O /tmp/qemu-image.sig
+  
https://ftp.gnu.org/gnu/guix/guix-system-vm-image-1.3.0.x86_64-linux.qcow2.sig
+wget https://sv.gnu.org/people/viewgpg.php?user_id=127547 \
+  -qO - | gpg --import -
+gpg --verify /tmp/qemu-image.sig
+```
+
+You can then spawn the VM, using [QEMU](https://qemu.org) (with network
+access and at least 2 GiB or RAM):
+
+```
+qemu-system-x86_64 \
+  -nic user,model=virtio-net-pci \
+  -enable-kvm -m 2048 \
+  -device virtio-blk,drive=myhd \
+  -drive if=none,file=/tmp/qemu-image,id=myhd
+```
+
+Please see the installation instructions mentioned above for more
+information.
+
+# Contents of the Virtual Machine Image
+
+The image is not specific to this artifact evaluation: it is a “live”
+image of Guix System distributed as part of the latest Guix release
+(version 1.3.0 in this case).
+
+The live image boots in a graphical environment, Xfce.  You can start a
+terminal by clicking on the terminal icon at the bottom of the screen.
+
+The shell lets you run various commands, in particular the ‘guix’
+command and the usual GNU/Linux utilities.  You can also use it to
+install more packages.  For instance, you might install Emacs by
+running:
+
+```
+guix install emacs
+```
+
+> Note that most `guix` operations rely on good network connectivity to
+> download packages.  Failures due to transient networking issues are
+> harmless; operations such as `guix install` and `guix pull` are
+> transactional and can be restarted without any risks.
+
+# Supported Claims
+
+## Performance
+
+**Claim: our code authenticates between 600 and 700 commits per
+second.**
+
+The image runs version 1.3.0 of Guix, released in May 2020.  You can
+update it by running:
+
+```
+guix pull --commit=20303c0b1c75bc4770cdfa8b8c6b33fd6e77c168
+```
+
+This will first make a local clone of the [Git repository of
+Guix](https://git.savannah.gnu.org/cgit/guix.git/), which can take
+several minutes; a progress bar is displayed during that process.
+
+This is followed by a message along these lines:
+
+```
+Authenticating channel 'guix', commits 9edb3f6 to 20303c0 (17,390 new 
commits)...
+```
+
+This message and the subsequent progress bar correspond to the
+authentication of commits between the [revision that is
+shipped](https://archive.softwareheritage.org/browse/revision/9edb3f66fd807b096b48283debdcddccfea34bad/)
+and the [target
+revision](https://archive.softwareheritage.org/browse/revision/20303c0b1c75bc4770cdfa8b8c6b33fd6e77c168/),
+a year and more than 17,000 commits later.  This process (starting from
+the “Authenticating …” message until the progress bar reaches 100%)
+should take less than a minute on recent hardware, consistent with
+Section 8.3 of the paper.
+
+The performance claim can also be verified by running `guix git
+authenticate` as will be shown below.
+
+## Commit Authentication
+
+**Claim: `guix pull` authenticates commits.**
+
+The Git authentication code, like the rest of Guix, is [free
+software](https://www.gnu.org/philosophy/free-sw.html).  It underwent [a
+public review process](https://issues.guix.gnu.org/22883) among Guix
+developers.  It is written in Scheme, a functional programming language
+of the Lisp family, and the current revision can be seen at:
+
+  - [core Git authentication
+    
module](https://archive.softwareheritage.org/browse/revision/20303c0b1c75bc4770cdfa8b8c6b33fd6e77c168/?path=guix/git-authenticate.scm)
+  - [authentication support for the “channel” mechanism, used by `guix
+    
pull`](https://archive.softwareheritage.org/browse/revision/20303c0b1c75bc4770cdfa8b8c6b33fd6e77c168/?path=guix/channels.scm#L324)
+  - [unit tests of the authentication
+    
module](https://archive.softwareheritage.org/browse/revision/20303c0b1c75bc4770cdfa8b8c6b33fd6e77c168/?path=tests/git-authenticate.scm)
+
+This code is used by the `guix` command provided in this virtual machine
+image.
+
+More informally, to experiment with the authentication mechanism, we
+invite reviewers to follow the steps below:
+
+  1. Install Git and GnuPG with `guix install git gnupg`.
+  2. Clone the Guix repository: `git clone
+     https://git.savannah.gnu.org/git/guix.git; cd guix`
+  3. Notice that commits are signed: `git log --oneline
+     --show-signature`.
+  4. With the command above, `gpg` will report display information about
+     each commit signature and report that public keys are unavailable
+     in its keyring (“No public key”).  Those keys are available in the
+     `keyring` branch and can be imported from there: `git checkout
+     keyring && cat *.key | gpg --import`.
+  5. Going back to our target revision, we can see that `gpg` can indeed
+     verify signatures now: `git checkout
+     20303c0b1c75bc4770cdfa8b8c6b33fd6e77c168 && git log --oneline
+     --show-signature`.  `gpg` warns about expired keys but, as the
+     paper notes, OpenPGP key expiration dates are ignored for our
+     purposes (and the authentication code in Guix does *not* use
+     `gpg`).
+
+We can now exercise the authentication code of Guix, this time using the
+`guix git authenticate` command, which is low-level compared to `guix
+pull` (it operates on raw Git repositories rather than on channels).  To
+authenticate the repository, run:
+
+```
+guix git authenticate \
+  9edb3f66fd807b096b48283debdcddccfea34bad \
+  "BBB0 2DDF 2CEA F6A8 0D1D  E643 A2A0 6DF2 A33A 54FA"
+```
+
+This authenticates the whole repository, starting from commit `9edb3f6`,
+which is expected to be signed by the OpenPGP key with fingerprint `BBB0
+2DDF …`.  This corresponds to what our `guix pull` command above did
+though `guix pull` was able to skip parent commits of its own commit (as
+returned by `guix describe`).  On success, `guix git authenticate`
+prints nothing and has exit code zero.  A cache of
+previously-authenticated commits is kept under
+`~/.cache/guix/authentication`.
+
+To see the effect of an unsigned commit on authentication, run:
+
+```
+git config --global user.email reviewer@example.org
+git config --global user.name Programming
+git rm README
+git commit -m "This is unsigned." -a
+guix git authenticate \
+  9edb3f66fd807b096b48283debdcddccfea34bad \
+  "BBB0 2DDF 2CEA F6A8 0D1D  E643 A2A0 6DF2 A33A 54FA"
+```
+
+This time, the command should fail with an error such as:
+
+```
+guix git: error: commit XYZ lacks a signature
+```
+
+One can similarly test the effect of a commit signed with an
+unauthorized key—a key whose fingerprint does not appear in the
+`.guix-authorizations` file.
+
+This behavior shown here is consistent with the claims in Section 4 of
+the paper.
+
+## Downgrade Prevention
+
+**Claim: `guix pull` cannot unwillingly downgrade `guix`.**
+
+If you ran `guix pull --commit=20303c0b1c75bc4770cdfa8b8c6b33fd6e77c168`
+as instructed above, the revision of Guix you are now using is commit
+20303c0…; `guix describe` should confirm that.
+
+Now let’s instruct `guix pull` to target a commit that is the parent of
+this one and is thus “older”:
+
+```
+guix pull --commit=b1e7e64f351fa03a66ce1f9776f9ba84cf2c6294
+```
+
+This command fails with:
+
+```
+guix pull: error: aborting update of channel 'guix' to commit
+b1e7e64f351fa03a66ce1f9776f9ba84cf2c6294, which is not a descendant of
+20303c0b1c75bc4770cdfa8b8c6b33fd6e77c168
+```
+
+An attempt to pull the `version-1.2.0` branch, which is older than the
+currently used commit, fails in a similar way:
+
+```
+guix pull --branch=version-1.2.0
+```
+
+This is consistent with claims in Section 6 of the paper.
+
+## System Downgrade Prevention
+
+**Claim: `guix system reconfigure` cannot unwillingly downgrade the system.**
+
+The paper further claims that `guix system reconfigure` similarly
+prevents downgrade *of the system*.  To verify this claim, we need to
+reconfigure (upgrade) the system running in the VM twice: first update
+it to commit 20303c0, then attempt to reconfigure it from an older
+commit, and notice that it fails.
+
+> Note: The first `reconfigure` operation is demanding in terms of disk
+> space and network bandwidth and takes several minutes, mostly
+> downloading the latest software packages.  It is necessary because the
+> system running in the VM lacks provenance information: `guix system
+> describe` does not show the commit that was used to build it.
+
+First, reconfigure the system:
+
+```
+sudo guix system reconfigure /run/current-system/configuration.scm
+```
+
+Upon completion, notice that the system provenance is indeed commit
+20303c0 as shown in the “channels” section of:
+
+```
+guix system describe
+```
+
+From there, attempt to reconfigure to an older commit (we use
+`time-machine` which fetches the given commit, similar to what `guix
+pull` does, and immediately launches, from that commit, the command that
+follows):
+
+```
+sudo guix time-machine \
+  --commit=b1e7e64f351fa03a66ce1f9776f9ba84cf2c6294 -- \
+  system reconfigure /run/current-system/configuration.scm
+```
+
+This command first builds the requested Guix revision, which can take a
+few minutes.  The subsequent `system reconfigure` fails:
+
+```
+guix system: error: aborting reconfiguration because commit
+b1e7e64f351fa03a66ce1f9776f9ba84cf2c6294 of channel 'guix' is not a
+descendant of 20303c0b1c75bc4770cdfa8b8c6b33fd6e77c168
+```
+
+This is consistent with claims in Section 6 of the paper.
+
+**Claim: Likewise, `guix deploy` cannot unwillingly downgrade the system
+on a set of machines.**
+
+We do not provide instructions to verify this claim because it requires
+a set of one or more machines running Guix System that we would remotely
+upgrade over SSH (secure shell), which is impractical to set up.
+[Downgrade prevention for `guix
+deploy`](https://archive.softwareheritage.org/browse/revision/20303c0b1c75bc4770cdfa8b8c6b33fd6e77c168/?path=gnu/machine/ssh.scm#L304)
+builds upon [downgrade prevention for `guix system
+reconfigure`](https://archive.softwareheritage.org/browse/revision/20303c0b1c75bc4770cdfa8b8c6b33fd6e77c168/?path=guix/scripts/system/reconfigure.scm#L309).