[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[PATCH v2 21/25] tests/docker: add --registry support to tooling
From: |
Alex Bennée |
Subject: |
[PATCH v2 21/25] tests/docker: add --registry support to tooling |
Date: |
Wed, 24 Jun 2020 15:04:42 +0100 |
This allows us to point the tools towards a registry from which they
can grab pre-built layers instead of doing everything from scratch
each time. To enable this we need to be using the DOCKER_BUILDKIT
engine.
[AJB: note registry.gitlab.com/stsquad/qemu is for my testing, the
final version will see DOCKER_REGISTRY default to
registry.gitlab.com/qemu-project/qemu]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
tests/docker/Makefile.include | 3 +++
tests/docker/docker.py | 25 +++++++++++++++++++++----
2 files changed, 24 insertions(+), 4 deletions(-)
diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index 7bac1a67e3e..4c5ebc6197f 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -13,6 +13,7 @@ DOCKER_IMAGES := $(sort $(notdir $(basename $(wildcard
$(DOCKER_FILES_DIR)/*.doc
DOCKER_TARGETS := $(patsubst %,docker-image-%,$(DOCKER_IMAGES))
# Use a global constant ccache directory to speed up repetitive builds
DOCKER_CCACHE_DIR := $$HOME/.cache/qemu-docker-ccache
+DOCKER_REGISTRY := $(if
$(REGISTRY),$(REGISTRY),registry.gitlab.com/stsquad/qemu)
DOCKER_TESTS := $(notdir $(shell \
find $(SRC_PATH)/tests/docker/ -name 'test-*' -type f))
@@ -57,6 +58,7 @@ docker-image-%: $(DOCKER_FILES_DIR)/%.docker
$(call quiet-command,\
$(DOCKER_SCRIPT) build -t qemu/$* -f $< \
$(if $V,,--quiet) $(if $(NOCACHE),--no-cache) \
+ $(if $(DOCKER_REGISTRY),--registry $(DOCKER_REGISTRY)) \
$(if $(NOUSER),,--add-current-user) \
$(if $(EXTRA_FILES),--extra-files $(EXTRA_FILES))\
$(if $(EXECUTABLE),--include-executable=$(EXECUTABLE)),\
@@ -213,6 +215,7 @@ endif
@echo ' Include extra files in image.'
@echo ' ENGINE=auto/docker/podman'
@echo ' Specify which container engine to run.'
+ @echo ' REGISTRY=url Cache builds from registry
(default:$(DOCKER_REGISTRY))'
# This rule if for directly running against an arbitrary docker target.
# It is called by the expanded docker targets (e.g. make
diff --git a/tests/docker/docker.py b/tests/docker/docker.py
index cc6f76caa60..09dedbf83f9 100755
--- a/tests/docker/docker.py
+++ b/tests/docker/docker.py
@@ -221,6 +221,8 @@ class Docker(object):
""" Running Docker commands """
def __init__(self):
self._command = _guess_engine_command()
+ if "docker" in self._command:
+ os.environ["DOCKER_BUILDKIT"] = "1"
self._instance = None
atexit.register(self._kill_instances)
signal.signal(signal.SIGTERM, self._kill_instances)
@@ -289,10 +291,16 @@ class Docker(object):
return labels.get("com.qemu.dockerfile-checksum", "")
def build_image(self, tag, docker_dir, dockerfile,
- quiet=True, user=False, argv=None, extra_files_cksum=[]):
+ quiet=True, user=False, argv=None, registry=None,
+ extra_files_cksum=[]):
if argv is None:
argv = []
+ if registry is not None:
+ dockerfile = dockerfile.replace("FROM qemu/",
+ "FROM %s/qemu/" %
+ (registry))
+
tmp_df = tempfile.NamedTemporaryFile(mode="w+t",
encoding='utf-8',
dir=docker_dir, suffix=".docker")
@@ -313,8 +321,14 @@ class Docker(object):
tmp_df.flush()
- self._do_check(["build", "-t", tag, "-f", tmp_df.name] + argv +
- [docker_dir],
+ build_args = ["build", "-t", tag, "-f", tmp_df.name]
+ if registry is not None:
+ cache = "%s/%s" % (registry, tag)
+ build_args += ["--cache-from", cache]
+ build_args += argv
+ build_args += [docker_dir]
+
+ self._do_check(build_args,
quiet=quiet)
def update_image(self, tag, tarball, quiet=True):
@@ -403,6 +417,8 @@ class BuildCommand(SubCommand):
parser.add_argument("--add-current-user", "-u", dest="user",
action="store_true",
help="Add the current user to image's passwd")
+ parser.add_argument("--registry", "-r",
+ help="cache from docker registry")
parser.add_argument("-t", dest="tag",
help="Image Tag")
parser.add_argument("-f", dest="dockerfile",
@@ -458,7 +474,8 @@ class BuildCommand(SubCommand):
for k, v in os.environ.items()
if k.lower() in FILTERED_ENV_NAMES]
dkr.build_image(tag, docker_dir, dockerfile,
- quiet=args.quiet, user=args.user, argv=argv,
+ quiet=args.quiet, user=args.user,
+ argv=argv, registry=args.registry,
extra_files_cksum=cksum)
rmtree(docker_dir)
--
2.20.1
- [PATCH v2 11/25] tests/vm: Add workaround to consume console, (continued)
- [PATCH v2 11/25] tests/vm: Add workaround to consume console, Alex Bennée, 2020/06/24
- [PATCH v2 05/25] tests/vm: Added configuration file support, Alex Bennée, 2020/06/24
- [PATCH v2 07/25] tests/vm: Added a new script for ubuntu.aarch64., Alex Bennée, 2020/06/24
- [PATCH v2 08/25] tests/vm: Added a new script for centos.aarch64., Alex Bennée, 2020/06/24
- [PATCH v2 10/25] python/qemu: Add ConsoleSocket for optional use in QEMUMachine, Alex Bennée, 2020/06/24
- [PATCH v2 09/25] tests/vm: change scripts to use self._config, Alex Bennée, 2020/06/24
- [PATCH v2 12/25] tests/vm: switch from optsparse to argparse, Alex Bennée, 2020/06/24
- [PATCH v2 16/25] .gitignore: un-ignore .gitlab-ci.d, Alex Bennée, 2020/06/24
- [PATCH v2 14/25] tests/docker: check for an parameters not empty string, Alex Bennée, 2020/06/24
- [PATCH v2 15/25] tests/docker: change tag naming scheme of our images, Alex Bennée, 2020/06/24
- [PATCH v2 21/25] tests/docker: add --registry support to tooling,
Alex Bennée <=
- [PATCH v2 13/25] tests/vm: allow us to take advantage of MTTCG, Alex Bennée, 2020/06/24
- [PATCH v2 24/25] gitlab: add avocado asset caching, Alex Bennée, 2020/06/24
- [PATCH v2 23/25] gitlab: enable check-tcg for linux-user tests, Alex Bennée, 2020/06/24
- [PATCH v2 20/25] gitlab: build containers with buildkit and metadata, Alex Bennée, 2020/06/24
- [PATCH v2 19/25] gitlab: convert jobs to use custom built containers, Alex Bennée, 2020/06/24
- [PATCH v2 17/25] gitlab: introduce explicit "container" and "build" stages, Alex Bennée, 2020/06/24
- [PATCH v2 25/25] gitlab: split build-disabled into two phases, Alex Bennée, 2020/06/24
- [PATCH v2 22/25] gitlab: add acceptance testing to system builds, Alex Bennée, 2020/06/24
- [PATCH v2 18/25] gitlab: build all container images during CI, Alex Bennée, 2020/06/24