Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)

[Ani Sinha]

On Fri, Jul 1, 2022 at 1:08 PM Michael S. Tsirkin <mst@redhat.com> wrote:
>
> On Fri, Jul 01, 2022 at 12:58:33PM +0530, Ani Sinha wrote:
> > On Fri, Jul 1, 2022 at 12:23 PM Michael S. Tsirkin <mst@redhat.com> wrote:
> > >
> > > On Fri, Jul 01, 2022 at 06:12:14AM +0200, Thomas Huth wrote:
> > > > I even wouldn't mind if you put your python stuff in a new directory 
> > > > like
> > > > tests/pytests/ for example, as long as it downloads your binaries 
> > > > separately
> > > > - as I wrote in another mail, the avocado framework rather looks like an
> > > > oddball in our test framework nowadays since it uses a separate test 
> > > > runner
> > > > and not the meson test harness, so having a new approach for 
> > > > python-based
> > > > tests is maybe even a good idea. I just really want to avoid that this 
> > > > goes
> > > > into tests/qtest (since it really does not belong there), and please 
> > > > don't
> > > > add more external stuff via git submodules, that's really the wrong 
> > > > approach
> > > > for this.
> > >
> > > I get it, people hate submodules with passion.  I think trying another
> > > approach for testing that is neither avocado nor qtest is
> > > not too bad. As long as this is not user visible, we can
> > > allow ourselves space to experiment.
> > >
> > > OK so, how about this:
> > > - put it in a new directory: tests/roms?
> > > - create repo for a fork of biosbits under git.qemu.org
> > > - roll our own analog to git submodules: a script
> > >   that clones the repo
> >
> > No need to clone the whole repo. We can simply download the binaries
> > that the girlab CI job would generate from the bits sources in that
> > repo.
> > We need to clone if we are always building bits from source for every
> > test. That is not necessary IMHO since much of the bits package would
> > remain as is without modification.
>
> IMHO CI job idea isn't great since isn't versioned at all, is it?
> Also as long as test passes, sure. But if it fails one will
> need the sources to investigate.
>
> Let's start with building things from source. Add an option
> of prebuilt binaries as an optimization once things
> stabilize.
>
>
> > > - new target make check-roms,
> >
> > I think make pytest or some such is better and more generic if other
> > such tests in other areas follow suit.
>
> The name is not critical in my mind, but I think we need to decide
> what exactly differentiates it from other tests.
>
>
> >
> > if the clone exists locally -
> > >   run the test, if not - skip it
> >
> > if download of the bits binaries fail, skip it.
>
> You seem to be recreating either git or avocado or both here.
>
> Personally I want something that works offline.

Well we need to make a call as to how to make things work offline, if
that's what we wanted.  I have made changes in v2 to download the
binaries from my github repo. It works great. If download fails, it
skips the test. If those files are available before, it will not
attempt to download them.
https://pastebin.com/nrcdN8WS and
https://pastebin.com/m7Dx3PZk

The Dockerfile and build script I have in the repo will generate these
artifacts. So if you wanted to run the test offline, either build
those from source and place them where the test expects to find them
or build them from source using Dockerfile/build script and place them
there before running the test.

>
>
>
> > > - as for using pre-generates ISOs as an optimization,
> > >   I'm not sure how important that is, if yes -
> > >   we can add another repo and another make target along the
> > >   same lines
> > >
> > >
> > >
> > > --
> > > MST
> > >
>