Re: [PATCH v2 10/11] pytest: add pytest to the meson build system

[Ani Sinha]

On Mon, 11 Jul 2022, John Snow wrote:

> On Sun, Jul 10, 2022 at 1:01 PM Ani Sinha <ani@anisinha.ca> wrote:
> >
> > Integrate the pytest framework with the meson build system. This will make 
> > meson
> > run all the pytests under the pytest directory.
> >
> > Signed-off-by: Ani Sinha <ani@anisinha.ca>
> > ---
> >  tests/Makefile.include   |  4 +++-
> >  tests/meson.build        |  1 +
> >  tests/pytest/meson.build | 49 ++++++++++++++++++++++++++++++++++++++++
> >  3 files changed, 53 insertions(+), 1 deletion(-)
> >  create mode 100644 tests/pytest/meson.build
> >
> > diff --git a/tests/Makefile.include b/tests/Makefile.include
> > index 3accb83b13..40755a6bd1 100644
> > --- a/tests/Makefile.include
> > +++ b/tests/Makefile.include
> > @@ -3,12 +3,14 @@
> >  .PHONY: check-help
> >  check-help:
> >         @echo "Regression testing targets:"
> > -       @echo " $(MAKE) check                  Run block, qapi-schema, 
> > unit, softfloat, qtest and decodetree tests"
> > +       @echo " $(MAKE) check                  Run block, qapi-schema, 
> > unit, softfloat, qtest, pytest and decodetree tests"
>
> Does this mean that "make check" *requires* an internet connection?

No. My test will be skipped if it is unable to download the artifacts it
requires due to lack of Internet connectivity.


If
> so, I'm afraid that introduces some complications for downstreams
> which require that "make check" can run without an internet
> connection. It's something I've been trying to wrestle with as I split
> the qemu.qmp library out of the QEMU tree, and I have been working
> (slowly) on remedying it with some additional Makefile logic.
>
> I have been looking into making a testing "dummy package" for python
> with optional dependency groups that use a venv building script to
> either pull from PyPI in online mode, or use system packages in
> offline mode. In the case of offline mode, I am working on augmenting
> the configure script to test that dependencies are met, and disabling
> test groups when the correct dependencies cannot be located.
>
> I hope to have another version of that series soon; it should be
> trivial to add a new optional dependency group to it. I'll CC you on
> the next version of the series.
>
> --js
>
> >         @echo " $(MAKE) bench                  Run speed tests"
> >         @echo
> >         @echo "Individual test suites:"
> >         @echo " $(MAKE) check-qtest-TARGET     Run qtest tests for given 
> > target"
> >         @echo " $(MAKE) check-qtest            Run qtest tests"
> > +       @echo " $(MAKE) check-pytest           Run pytest tests"
> > +       @echo " $(MAKE) check-pytest-TARGET    Run pytest for a given 
> > target"
> >         @echo " $(MAKE) check-unit             Run qobject tests"
> >         @echo " $(MAKE) check-qapi-schema      Run QAPI schema tests"
> >         @echo " $(MAKE) check-block            Run block tests"
> > diff --git a/tests/meson.build b/tests/meson.build
> > index 8e318ec513..f344cbdc6c 100644
> > --- a/tests/meson.build
> > +++ b/tests/meson.build
> > @@ -93,3 +93,4 @@ subdir('unit')
> >  subdir('qapi-schema')
> >  subdir('qtest')
> >  subdir('migration')
> > +subdir('pytest')
> > diff --git a/tests/pytest/meson.build b/tests/pytest/meson.build
> > new file mode 100644
> > index 0000000000..e60d481ae4
> > --- /dev/null
> > +++ b/tests/pytest/meson.build
> > @@ -0,0 +1,49 @@
> > +slow_pytests = {
> > +  'acpi-bits' : 120,
> > +}
> > +
> > +pytests_generic = []
> > +
> > +# biosbits tests are currenly only supported on x86_64 platforms.
> > +pytests_x86_64 = ['acpi-bits-test']
> > +
> > +pytest_executables = {}
> > +other_deps = []
> > +
> > +subdir('acpi-bits')
> > +
> > +foreach dir : target_dirs
> > +  if not dir.endswith('-softmmu')
> > +    continue
> > +  endif
> > +
> > +  target_base = dir.split('-')[0]
> > +  pytest_emulator = emulators['qemu-system-' + target_base]
> > +  target_pytests = get_variable('pytests_' + target_base, []) + 
> > pytests_generic
> > +
> > +  test_deps = roms
> > +  pytest_env = environment()
> > +  if have_tools
> > +    pytest_env.set('QTEST_QEMU_IMG', './qemu-img')
> > +    test_deps += [qemu_img]
> > +  endif
> > +  pytest_env.set('G_TEST_DBUS_DAEMON', meson.project_source_root() / 
> > 'tests/dbus-vmstate-daemon.sh')
> > +  pytest_env.set('PYTEST_QEMU_BINARY', './qemu-system-' + target_base)
> > +  pytest_env.set('PYTEST_SOURCE_ROOT', meson.project_source_root())
> > +  if have_tools and have_vhost_user_blk_server
> > +    pytest_env.set('PYTEST_QEMU_STORAGE_DAEMON_BINARY', 
> > './storage-daemon/qemu-storage-daemon')
> > +    test_deps += [qsd]
> > +  endif
> > +
> > +  foreach test : target_pytests
> > +    test('pytest-@0@/@1@'.format(target_base, test),
> > +         pytest_executables[test],
> > +         depends: [test_deps, pytest_emulator, emulator_modules, 
> > other_deps],
> > +         env: pytest_env,
> > +         args: ['--tap', '-k'],
> > +         protocol: 'tap',
> > +         timeout: slow_pytests.get(test, 30),
> > +         priority: slow_pytests.get(test, 30),
> > +         suite: ['pytest', 'pytest-' + target_base])
> > +  endforeach
> > +endforeach
> > --
> > 2.25.1
> >
>
>