19/66: icse-2022: Add CCS reviews.

[Ludovic Courtès]

civodul pushed a commit to branch master
in repository maintenance.

commit a7ee36f9740a6f3fa94d5b155772a316ce258b18
Author: Ludovic Courtès <ludo@gnu.org>
AuthorDate: Tue Aug 31 12:14:36 2021 +0200

    icse-2022: Add CCS reviews.
---
 doc/icse-2022/ccs-reviews.txt | 173 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 173 insertions(+)

diff --git a/doc/icse-2022/ccs-reviews.txt b/doc/icse-2022/ccs-reviews.txt
new file mode 100644
index 0000000..3cc9eb4
--- /dev/null
+++ b/doc/icse-2022/ccs-reviews.txt
@@ -0,0 +1,173 @@
+From: "ACM CCS 2021 B HotCRP" <noreply@ccs2021-b.hotcrp.com>
+Subject: [ACM CCS 2021 B] Submission #142 "Building a Secure Software Supply 
Chain..."
+To: Ludovic Courtès <ludovic.courtes@inria.fr>
+Date: Wed,  9 Jun 2021 23:59:20 +0000 (UTC)
+Reply-To: ccs2021@cs.ucsb.edu
+
+Dear authors,
+
+The ACM Conference on Computer and Communications Security (ACM CCS
+2021) program committee is sorry to inform you that your submission #142
+was rejected after the first round of reviews, and will not appear in
+the conference.
+
+       Title: Building a Secure Software Supply Chain with GNU Guix
+        Site: https://ccs2021-b.hotcrp.com/paper/142
+
+If you have any questions please feel free to contact the Program
+Co-Chairs.
+
+Best regards,
+ Giovanni Vigna and Elaine Shi
+
+Review #142A
+===========================================================================
+
+Overall merit
+-------------
+2. Weak reject
+
+Reviewer expertise
+------------------
+3. Knowledgeable
+
+Paper summary
+-------------
+This paper describes the rationale and design behind the GNU Guix package and 
system manager,
+and then zooms in on the design of a new security feature that supports secure 
updates with
+Guix. More specifically, the paper describes an approach for authenticating 
git checkouts, based on the notion
+of an authorization invariant that is checked with every pull from a 
repository.
+The paper describes the attacks that are stopped with this new security 
extension of Guix, and
+describes an implementation that is already in practical use.
+
+Strengths
+---------
++ this was an interesting paper! This reviewer was not familiar with GNU Guix, 
and the paper 
+gives a very well-motivated and well-explained account of the system
+
++ addressing the issue of supply-chain attacks is very important and 
challenging
+
++ the paper describes a working system that seems to be in real use today
+
+Weaknesses
+----------
+- this is more a high-level design description or a technical white paper, not 
a scientific paper
+
+Comments for author
+-------------------
+I very much enjoyed reading this paper. I found the introduction to Guix in 
the first two pages of 
+the paper to be well-written and to make a good case for the technical 
benefits of Guix with 
+respect to protecting the software supply chain. The combination of a 
functional deployment
+model, reproducible builds, and a very strong reduction of the size of 
"bootstrap binaries" leads
+to a conceptually clean system.
+I also think the problem that the paper addresses (improving the security of 
updates by
+authenticating git checkouts) is well explained, and the design described in 
the paper, based
+on the central idea of an "authentication invariant" makes sense. The 
authentication invariant
+requires that a commit is signed by one of the keys listed in a specific file 
in each of the parents
+of the commit.
+So there is a lot of positive things to say about this paper.
+However, a major showstopper is that the paper provides no scientific 
evaluation of the proposed
+design. The paper does report on experience in an almost anecdotal way, using 
statements like:
+
+- on performance evaluation: "As an indication, on a recent x86_64 laptop 
(Intel i7 CPU
+at 2.6 GHz with data stored on a solid state device, SSD),
+our code authenticates between 600 and 700 commits per
+second, which is fast enough to not be detrimental to the
+user experience"
+
+- on usability evaluation: "Some users were initially surprised but quickly 
understood the value of
+this protection."
+
+This kind of evaluation is, in the opinion of this reviewer, too informal for 
a scientific conference
+like ACM CCS, and hence I am not in favor of accepting the paper. I do believe 
it deserves to
+be published though, but in a different venue (for instance a journal like 
;login published by
+Usenix whose goal is to disseminate information about useful research and 
tools to the wide
+Usenix community).
+
+
+* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
+
+
+Review #142B
+===========================================================================
+
+Overall merit
+-------------
+1. Reject
+
+Reviewer expertise
+------------------
+3. Knowledgeable
+
+Paper summary
+-------------
+This paper deals with (software) supply chain attacks. More specifically, the
+authors present the design (along with implementation details) of GNU Guix: a
+package management system the enables reproducible builds and provenance
+tracking (of the relevant code bits). Guix has been around almost ~8y and the
+authors focus on the latest advances of the tool re: source code
+authentication.
+
+The paper begins with an overview of the whole problem of reproducible builds,
+followed by the approach taken by Guix. Next, the authors present the gist of
+their idea, which is basically a scheme for authenticated Git checkouts.  More
+specifically, commits are signed and the respective repositories contain a file
+(`.guix-authorizations`) that includes the authorized committers. Upon a
+checkout the package manager authenticates all commits by verifying the
+corresponding signatures and performing lookups against `guix-authorizations`.
+
+Given the aforementioned building block the authors also describe how Guix can
+defend against _downgrade attacks_ as well as attacks where the victim is
+redirected to a different repository/branch (_teleport attack_).
+
+Lastly, as far as the implementation goes, the authors used OpenPGP signatures
+for signing and authenticating Git commits, while a benchmarks was also
+performed on a contemporary host (Intel Core i7 CPU), which demonstrated the
+ability to authenticate 600--700 commits per sec.
+
+Strengths
+---------
+* [+] Interesting problem.
+* [+] Integration into a real-world codebase.
+
+Weaknesses
+----------
+* [-] Novelty.
+* [-] The contribution(s) is/are unclear.
+
+Comments for author
+-------------------
+Overall, the paper was fun to read and focuses on a realistic threat: that is,
+software supply chain attacks. Guix is a great tool for reproducible builds and
+provenance tracking, and it's nice to know that it takes steps to further
+shield itself against downgrade attacks, teleport attacks, _etc._
+
+However, the paper basically reads as a technical report for the latest
+capabilities added to Guix re: Git commit authentication. In addition, even
+that is straightforward: the commits are signed, using OpenPGP signatures, and
+the package manager basically verifies signatures and performs lookups against
+a list of authorized committers.
+
+So, as much I appreciate Guix taking steps towards secure provenance, there is
+very little scientific value in the current paper. What is the hypothesis
+you're trying to (dis)prove? The fact that we can authenticate Git commits
+using public-private crypto? This is something already well-known, and so the
+paper is basically an exercise re: how to properly implement such a scheme.
+
+If there is a scientific question that the authors are trying to answer, then
+I'd suggest to revise their work and frame the respective questions and
+findings accordingly. If not, then a paper (can potentially) still have merits
+if it advances the state-of-the-art in terms of a particular _technical_
+problem. If that's the case, then again I'd recommend that the authors describe
+the technical challenges involved and why/how their solution advances the
+state-of-the-art -- in terms of research, not in terms of a particular code
+base only.
+
+Questions for authors’ response
+---------------------------------
+See _"Comments for author"_ above.
+
+Concerns to be addressed during revision/shepherding
+----------------------------------------------------
+See _"Comments for author"_ above.
+