[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[RFC PATCH v3 0/7] tests: run python tests under a venv
From: |
John Snow |
Subject: |
[RFC PATCH v3 0/7] tests: run python tests under a venv |
Date: |
Mon, 11 Jul 2022 19:01:48 -0400 |
Hi, here's another RFC for bringing external Python dependencies to the
QEMU test suite.
This is mostly a refresh of a version I sent out before, but mixes in my
VM improvement test as an optional pre-requisite to improve VM test
stability to ensure that the BSDs all pass with the new
infrastructure. (And our oldest supported Debian and Ubuntu targets,
too.)
(Note: this requires dropping support for Ubuntu 18.04, which ships with
a version of setuptools that is simply too old.)
This patchset is still not without some problems that I am working on,
but progress has been slow.
Problems I am aware of:
- This version of the patch series does not itself enforce any
offline-only behavior for venv creation, but downstreams can modify
any call to 'mkvenv' to pass '--offline'. I am working on a configure
file toggle to swap the default behavior when running tests.
- iotests will now actually never run mypy or pylint tests by default
anymore, because the bootstrapper won't select those packages by
default, and the virtual environment won't utilize the system
packages, so iotest 297 will just "skip" all of the time now.
The reason we don't want to install these packages by default is
because we don't want to add dependencies on mypy and pylint for
downstream builds.
With these patches, 297 would still work if you manually opened up the
testing venv and installed suitable mypy/pylint packages there. I
could also add a new optional dependency group, and one could
theoretically invoke a once-per-build-dir command of 'make
check-venv-iotests' to help make the process only semi-manual, but
it's still annoying.
Ideally, the python checks in qemu.git/python/ can handle the same
tests as 297 does -- but we need to give a shorthand invocation like
"make check-python" that is excluded from the default "make check" to
allow block developers to quickly opt-in to the same tests.
I've covered some of the problems here on-list before:
https://lists.gnu.org/archive/html/qemu-devel/2022-05/msg03661.html
...But I haven't quite solved them yet.
- iotests can now self-bootstrap the venv when it isn't present, but
this self-bootstrapping has some issues in that because it skips the
Makefile magic, it cannot update the venv when changes are made to the
venv configuration piece.
That's all for now. Work on the configure file integration is ongoing. I
don't know if I'll beat soft freeze (It's looking unlikely given the
amount of IRL issues I am juggling right now), but I'm hoping to push as
much of this forward as I can to try and get some testing in for RC
phase to determine what problems might exist that I haven't noticed yet.
--js
John Snow (7):
tests: create optional tests/venv dependency groups
tests: pythonize test venv creation
tests: Remove spurious pip warnings on Ubuntu20.04
tests/vm: add venv pre-requisites to VM building recipes
tests: add 'check-venv' as a dependency of 'make check'
iotests: use tests/venv for running tests
iotests: self-bootstrap testing venv
tests/Makefile.include | 32 +++---
tests/mkvenv.py | 187 ++++++++++++++++++++++++++++++++++
tests/qemu-iotests/testenv.py | 25 +++--
tests/requirements.txt | 6 --
tests/setup.cfg | 20 ++++
tests/setup.py | 16 +++
tests/vm/netbsd | 1 +
7 files changed, 262 insertions(+), 25 deletions(-)
create mode 100644 tests/mkvenv.py
delete mode 100644 tests/requirements.txt
create mode 100644 tests/setup.cfg
create mode 100644 tests/setup.py
--
2.34.3
- [RFC PATCH v3 0/7] tests: run python tests under a venv,
John Snow <=
- [RFC PATCH v3 2/7] tests: pythonize test venv creation, John Snow, 2022/07/11
- [RFC PATCH v3 7/7] iotests: self-bootstrap testing venv, John Snow, 2022/07/11
- [RFC PATCH v3 6/7] iotests: use tests/venv for running tests, John Snow, 2022/07/11
- [RFC PATCH v3 4/7] tests/vm: add venv pre-requisites to VM building recipes, John Snow, 2022/07/11
- [RFC PATCH v3 1/7] tests: create optional tests/venv dependency groups, John Snow, 2022/07/11
- [RFC PATCH v3 5/7] tests: add 'check-venv' as a dependency of 'make check', John Snow, 2022/07/11
- [RFC PATCH v3 3/7] tests: Remove spurious pip warnings on Ubuntu20.04, John Snow, 2022/07/11