Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)

[Ani Sinha]

On Fri, Jul 1, 2022 at 1:08 PM Michael S. Tsirkin <mst@redhat.com> wrote:
>
> On Fri, Jul 01, 2022 at 12:58:33PM +0530, Ani Sinha wrote:
> > On Fri, Jul 1, 2022 at 12:23 PM Michael S. Tsirkin <mst@redhat.com> wrote:
> > >
> > > On Fri, Jul 01, 2022 at 06:12:14AM +0200, Thomas Huth wrote:
> > > > I even wouldn't mind if you put your python stuff in a new directory 
> > > > like
> > > > tests/pytests/ for example, as long as it downloads your binaries 
> > > > separately
> > > > - as I wrote in another mail, the avocado framework rather looks like an
> > > > oddball in our test framework nowadays since it uses a separate test 
> > > > runner
> > > > and not the meson test harness, so having a new approach for 
> > > > python-based
> > > > tests is maybe even a good idea. I just really want to avoid that this 
> > > > goes
> > > > into tests/qtest (since it really does not belong there), and please 
> > > > don't
> > > > add more external stuff via git submodules, that's really the wrong 
> > > > approach
> > > > for this.
> > >
> > > I get it, people hate submodules with passion.  I think trying another
> > > approach for testing that is neither avocado nor qtest is
> > > not too bad. As long as this is not user visible, we can
> > > allow ourselves space to experiment.
> > >
> > > OK so, how about this:
> > > - put it in a new directory: tests/roms?
> > > - create repo for a fork of biosbits under git.qemu.org
> > > - roll our own analog to git submodules: a script
> > >   that clones the repo
> >
> > No need to clone the whole repo. We can simply download the binaries
> > that the girlab CI job would generate from the bits sources in that
> > repo.
> > We need to clone if we are always building bits from source for every
> > test. That is not necessary IMHO since much of the bits package would
> > remain as is without modification.
>
> IMHO CI job idea isn't great since isn't versioned at all, is it?

bits is versioned yes, in a crude way. every time you make a commit in
the top level repo, the version would increment by one.

> Also as long as test passes, sure. But if it fails one will
> need the sources to investigate.

sources might also be needed to write the tests.

>
> Let's start with building things from source.

hmm. bitys uses old autotools, not ninja and takes about 10/15 mins to
build depending on parallelity and build host.

Add an option
> of prebuilt binaries as an optimization once things
> stabilize.
>
>
> > > - new target make check-roms,
> >
> > I think make pytest or some such is better and more generic if other
> > such tests in other areas follow suit.
>
> The name is not critical in my mind, but I think we need to decide
> what exactly differentiates it from other tests.
>
>
> >
> > if the clone exists locally -
> > >   run the test, if not - skip it
> >
> > if download of the bits binaries fail, skip it.
>
> You seem to be recreating either git or avocado or both here.
>
> Personally I want something that works offline.
>
>
>
> > > - as for using pre-generates ISOs as an optimization,
> > >   I'm not sure how important that is, if yes -
> > >   we can add another repo and another make target along the
> > >   same lines
> > >
> > >
> > >
> > > --
> > > MST
> > >
>