Re: Adjustments to Docker related packages and service

[Efraim Flashner]

On Sat, Sep 19, 2020 at 03:03:29PM -0400, Jesse Dowell wrote:
> Hello Guix,
> 
> I've been maintaining a set of patches to Docker in Guix that fix/modify
> various aspects of the package and service in ways that are helpful to me.
> 
> I'd like to get these changes upstream if possible but I don't know which
> of these changes might be qualified as a patch vs a bug fix. Also, it's
> possible that I'm unaware of side-effects that these changes might have on
> use-cases outside of my own scope of work.

I'm glad you want to upstream these, having more eyes on the packages
can only make them better. Also sometimes we end up using the packages
without really knowing if they're configured and built
correctly/optimally.

> The patches are attached. I think that each patch might stand independently
> on its own but I haven't tested them in that configuration.
> 
> A quick summary of each patch....

I'm going to make my comments inline in the patches.

> 
> 0001-gnu-docker-Synchronize-docker-dependencies-with-the-.patch
> 
> This patch synchronizes docker-libnetwork and go-sctp with the current
> version of docker. Without this patch, I've encountered some issues (I
> forget the specifics offhand - sorry). Also, I believe this is an
> appropriate change based on comments in the guix source code for the
> packages and build instructions I've seen elsewhere.
> 
> 0002-gnu-docker-compile-docker-with-seccomp-enabled.patch
> 
> I utilize docker seccomp profiles and the current version of docker on guix
> does not support them. This seems safe enough to enable for everyone but
> perhaps there's something I'm overlooking?
> 
> 0003-services-docker-shepherd-service-fix-enable-proxy-op.patch
> 
> This is a bug fix. I don't think the current code properly disables the
> userland-proxy when the option is set to false.
> 
> 0004-gnu-docker-use-nftables-via-iptables-legacy.patch
> 
> I'm guessing this patch won't fly as-is but it's helpful for systems using
> nftables instead of iptables. I could try to adjust it to make it flexible
> if there's any interest.
> 
> That's everything. Any suggestions or comments would be much appreciated.
> 
> Best,
> Jesse

> From 53e80970f69f2e15be8d320a8e1995ffd0bd8cb0 Mon Sep 17 00:00:00 2001
> From: Jesse Dowell <jessedowell@gmail.com>
> Date: Sat, 19 Sep 2020 12:08:27 -0400
> Subject: [PATCH 1/4] gnu: docker: Synchronize docker dependencies with the
>  current version of docker.
> 

I added a copyright line for you to these two files and I adjusted the
commit message to mention all of the changes.

> ---
>  gnu/packages/docker.scm     | 17 ++++++++++-------
>  gnu/packages/networking.scm |  8 +++++---
>  2 files changed, 15 insertions(+), 10 deletions(-)
> 
> diff --git a/gnu/packages/docker.scm b/gnu/packages/docker.scm
> index bc7e02ec46..69905e59bb 100644
> --- a/gnu/packages/docker.scm
> +++ b/gnu/packages/docker.scm
> @@ -48,6 +48,8 @@
>    #:use-module (gnu packages version-control)
>    #:use-module (gnu packages virtualization))
>  
> +;; Note - when changing Docker versions it is important to update the 
> versions
> +;; of several associated packages (docker-libnetwork and go-sctp).
>  (define %docker-version "19.03.12")
>  
>  (define-public python-docker
> @@ -246,9 +248,10 @@ network attachments.")
>  (define docker-libnetwork
>    ;; There are no recent release for libnetwork, so choose the last commit of
>    ;; the branch that Docker uses, as can be seen in the Docker source file
> -  ;; 'hack/dockerfile/install/proxy.installer'.
> -  (let ((commit "4725f2163fb214a6312f3beae5991f838ec36326")
> -        (version "18.09")
> +  ;; 'hack/dockerfile/install/proxy.installer'. NOTE - It is important that
> +  ;; this version is kept in sync with the version of Docker being used.
> +  (let ((commit "026aabaa659832804b01754aaadd2c0f420c68b6")
> +        (version "19.03")

this I replaced with (version-major+minor %docker-version)
I also added to the comment that it's from the 'bump_19.03' branch

>          (revision "1"))
>      (package
>        (name "docker-libnetwork")
> @@ -256,12 +259,12 @@ network attachments.")
>        (source (origin
>                  (method git-fetch)
>                  (uri (git-reference
> -                      (url "https://github.com/docker/libnetwork";)
> +                      (url "https://github.com/moby/libnetwork";)

I added mention of this to the commit message.

>                        (commit commit)))
>                  (file-name (git-file-name name version))
>                  (sha256
>                   (base32
> -                  "1zpnxki8qfzha6ljahpwd3vkzmjhsvkmf73w6crm4ilxxw5vnpfb"))
> +                  "0bli21vn5v7bssw3ydym4jfdjsldhb47fld88kng7d138wl70lkw"))
>                  ;; Delete bundled ("vendored") free software source code.
>                  (modules '((guix build utils)))
>                  (snippet '(begin
> @@ -269,8 +272,8 @@ network attachments.")
>                              #t))))
>        (build-system go-build-system)
>        (arguments
> -       `(#:import-path "github.com/docker/libnetwork/"))
> -      (home-page "https://github.com/docker/libnetwork/";)
> +       `(#:import-path "github.com/moby/libnetwork/"))
> +      (home-page "https://github.com/moby/libnetwork/";)

And here.

>        (synopsis "Networking for containers")
>        (description "Libnetwork provides a native Go implementation for
>  connecting containers.  The goal of @code{libnetwork} is to deliver a robust
> diff --git a/gnu/packages/networking.scm b/gnu/packages/networking.scm
> index 1c2e7cf979..964701af1a 100644
> --- a/gnu/packages/networking.scm
> +++ b/gnu/packages/networking.scm
> @@ -1435,8 +1435,10 @@ handling network namespaces in Go.")
>        (license license:asl2.0))))
>  
>  (define-public go-sctp
> -  ;; docker-libnetwork-cmd-proxy requires this exact commit.
> -  (let ((commit "07191f837fedd2f13d1ec7b5f885f0f3ec54b1cb")
> +  ;; docker-libnetwork-cmd-proxy requires this exact commit.  NOTE - It is
> +  ;; important that this version is kept in sync with the current version of
> +  ;; Docker and its associated dependencies.
> +  (let ((commit "6e2cb1366111dcf547c13531e3a263a067715847")
>          (revision "1"))

Since we changed the commit I also bumped the revision so it'll show up
as a new version. I also disabled the test suite since it was flakey on
the machines I tested with.

>      (package
>        (name "go-sctp")
> @@ -1449,7 +1451,7 @@ handling network namespaces in Go.")
>                  (file-name (git-file-name name version))
>                  (sha256
>                   (base32
> -                  "1mk9ncm10gwi5pn5wcw4skbyf4qg7n5qdf1mim4gf3mrckvi6g6h"))))
> +                  "1ba90fmpdwxa1ba4hrsjhi3gfy3pwmz7x8amw1p5dc9p5a7nnqrb"))))
>        (build-system go-build-system)
>        (arguments
>         `(#:import-path "github.com/ishidawataru/sctp"))
> -- 
> 2.28.0
> 

> From e5ecc9051ae14172acbab1e8b81e038f51a7bdf0 Mon Sep 17 00:00:00 2001
> From: Jesse Dowell <jessedowell@gmail.com>
> Date: Sat, 19 Sep 2020 12:40:38 -0400
> Subject: [PATCH 3/4] services: docker-shepherd-service: fix enable-proxy?
>  option
> 
> The userland proxy option does not properly disable the userland proxy when
> set to false. Docker defaults to enabling the userland proxy if the option is
> unset on the command line.

Also added a copyright line for you here and adjusted the commit
message.

> ---
>  gnu/services/docker.scm | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/gnu/services/docker.scm b/gnu/services/docker.scm
> index a4eccc7928..db7fdd5f4e 100644
> --- a/gnu/services/docker.scm
> +++ b/gnu/services/docker.scm
> @@ -121,9 +121,9 @@ loop-back communications.")
>                             #$@(if debug?
>                                    '("--debug" "--log-level=debug")
>                                    '())
> -                           (if #$enable-proxy? "--userland-proxy" "")
> -                           "--userland-proxy-path" (string-append #$proxy
> -                                                                  
> "/bin/proxy")
> +                           #$@(if enable-proxy?
> +                                  '("--userland-proxy=true" (string-append 
> "--userland-proxy-path=" proxy "/bin/proxy"))
> +                                  '("--userland-proxy=false"))

Only change here was to wrap the lines so they tried to fit in under 80
characters.

>                             (if #$enable-iptables?
>                                 "--iptables"
>                                 "--iptables=false"))
> -- 
> 2.28.0
> 

> From 163e84b2dc3d13d716fa1db27ab93893a5419303 Mon Sep 17 00:00:00 2001
> From: Jesse Dowell <jessedowell@gmail.com>
> Date: Sat, 19 Sep 2020 12:34:47 -0400
> Subject: [PATCH 2/4] gnu: docker: compile docker with seccomp enabled
> 
> ---
>  gnu/packages/docker.scm | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/gnu/packages/docker.scm b/gnu/packages/docker.scm
> index 69905e59bb..825aa1ef0f 100644
> --- a/gnu/packages/docker.scm
> +++ b/gnu/packages/docker.scm
> @@ -517,6 +517,7 @@ built-in registry server of Docker.")
>               #t))
>           (replace 'configure
>             (lambda _
> +             (setenv "DOCKER_BUILDTAGS" "seccomp")

I looked around a bunch for this. It looks like the seccomp option was
added around 2016 and it is currently part of the default build profile
for the official docker packages. It is also used for Debian. The other
two parts from upstream are apparmor and selinux, neither of which Guix
uses natively (IIRC). Since the default is 'nothing' and upstream
suggests using it this seems like a safe change.

Also, it can be overridden in any case by selecting a different profile
or using the '--privileged' flag when using 'docker run'.

>               (setenv "DOCKER_GITCOMMIT" (string-append "v" ,%docker-version))
>               (setenv "VERSION" (string-append ,%docker-version "-ce"))
>               ;; Automatically use bundled dependencies.
> -- 
> 2.28.0
> 

> From ac3277477bda6741ff3a8af9530c2fd68e2bb062 Mon Sep 17 00:00:00 2001
> From: Jesse Dowell <jessedowell@gmail.com>
> Date: Sat, 19 Sep 2020 12:45:39 -0400
> Subject: [PATCH 4/4] gnu: docker: use nftables via iptables-legacy
> 

I think you meant nftables via iptables

> ---
>  gnu/packages/docker.scm | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/gnu/packages/docker.scm b/gnu/packages/docker.scm
> index 825aa1ef0f..322f00026f 100644
> --- a/gnu/packages/docker.scm
> +++ b/gnu/packages/docker.scm
> @@ -438,8 +438,8 @@ built-in registry server of Docker.")
>                    ("pvdisplay" "lvm2" "sbin/pvdisplay")
>                    ("blkid" "util-linux" "sbin/blkid")
>                    ("unpigz" "pigz" "bin/unpigz")
> -                  ("iptables" "iptables" "sbin/iptables")
> -                  ("iptables-legacy" "iptables" "sbin/iptables")
> +                  ("iptables" "iptables" "sbin/iptables-nft")
> +                  ("iptables-legacy" "iptables" "sbin/iptables-legacy")

I checked the iptables package. 'iptables' and 'iptables-legacy' are
both symlinks for 'xtables-legacy-multi'. There is another binary for
'iptables-nft' which is a symlink for 'xtables-nft-multi'. Checking
through the source there aren't actually any references to
iptables-legacy in the docker tarball (except in the Dockerfile).

Guix has services for both iptables and nftables. I'd rather not break
existing workflows by switching iptables in docker to nftables. Also I
don't know if it is possible to easily support both in the same package.

>                    ("ip" "iproute2" "sbin/ip"))
>  
>                   (substitute-Command*
> -- 
> 2.28.0
> 

I've pushed the first 3 patches and I'd love to have some other input
on the iptables/nftables bit in the 4th patch.

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

signature.asc
Description: PGP signature