Re: [RFC PATCH 8/8] tests/docker: Selective line reading by python script

[Philippe Mathieu-Daudé]

+Erik/Daniel

On Wed, Jul 27, 2022 at 6:37 PM Lucas Mateus Castro(alqotel)
<lucas.araujo@eldorado.org.br> wrote:
>
> Building some images failed on ppc64le because the dockerfile tried to
> install some packages that are only available in x86 and arm64, to solve
> this while still having those packages be available in those architectures
> a comment was put before the installation command to instruct the python
> script into ignoring those lines for some architectures (in this case
> ppc64le)
>
> Overall I'm not a big fan of the way I solved this problem, so I'd like
> to know if anyone has a better way to make these dockerfilse work in
> PPC64LE.
>
> For context the base images used here are available in PPC64LE but some
> of the packages installed are not (in alpine's case it's XEN, which is
> only available to x86 and ARM), so this patch create a ignore_list which
> is set on a per-architecture basis, and any packages in a dockerfile in
> this ignore_list will not be copied to the temporary dockerfile used in
> the docker command.

Shouldn't this be done on lcitool side?
(https://gitlab.com/libvirt/libvirt-ci/-/tree/master/lcitool)

> Signed-off-by: Lucas Mateus Castro(alqotel) <lucas.araujo@eldorado.org.br>
> ---
>  tests/docker/docker.py                 | 15 ++++++++++++---
>  tests/docker/dockerfiles/alpine.docker |  2 ++
>  2 files changed, 14 insertions(+), 3 deletions(-)
>
> diff --git a/tests/docker/docker.py b/tests/docker/docker.py
> index d0af2861b8..9b962d1c78 100755
> --- a/tests/docker/docker.py
> +++ b/tests/docker/docker.py
> @@ -14,6 +14,7 @@
>  import os
>  import sys
>  import subprocess
> +import platform
>  import json
>  import hashlib
>  import atexit
> @@ -207,8 +208,15 @@ def _read_qemu_dockerfile(img_name):
>
>  def _dockerfile_preprocess(df):
>      out = ""
> +    ignore_list = []
>      for l in df.splitlines():
> -        if len(l.strip()) == 0 or l.startswith("#"):
> +        if len(l.strip()) == 0:
> +            continue
> +        if l.startswith("#"):
> +            if len(l.split()) >= 3:
> +                if l.split()[1] == "ignore":
> +                    if platform.processor() in l.split()[2].split(','):
> +                        ignore_list += l.split()[3].split(',')
>              continue
>          from_pref = "FROM qemu/"
>          if l.startswith(from_pref):
> @@ -219,7 +227,8 @@ def _dockerfile_preprocess(df):
>              inlining = _read_qemu_dockerfile(l[len(from_pref):])
>              out += _dockerfile_preprocess(inlining)
>              continue
> -        out += l + "\n"
> +        if not any(x in l.split() for x in ignore_list):
> +            out += l + "\n"
>      return out
>
>
> @@ -330,7 +339,7 @@ def build_image(self, tag, docker_dir, dockerfile,
>          tmp_df = tempfile.NamedTemporaryFile(mode="w+t",
>                                               encoding='utf-8',
>                                               dir=docker_dir, 
> suffix=".docker")
> -        tmp_df.write(dockerfile)
> +        tmp_df.write(_dockerfile_preprocess(dockerfile))
>
>          if user:
>              uid = os.getuid()
> diff --git a/tests/docker/dockerfiles/alpine.docker 
> b/tests/docker/dockerfiles/alpine.docker
> index 2943a99730..5cec46d8f2 100644
> --- a/tests/docker/dockerfiles/alpine.docker
> +++ b/tests/docker/dockerfiles/alpine.docker
> @@ -6,6 +6,8 @@
>
>  FROM docker.io/library/alpine:edge
>
> +# Lines to by ignored when this file is read by the python script
> +# ignore ppc64le,ppc64 xen-dev
>  RUN apk update && \
>      apk upgrade && \
>      apk add \
> --
> 2.25.1
>