Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits

[Ani Sinha]

On Wed, Sep 28, 2022 at 12:28 PM Daniel P. Berrangé <berrange@redhat.com> wrote:
>
> On Tue, Sep 27, 2022 at 05:18:10PM -0400, Michael S. Tsirkin wrote:
> > On Tue, Sep 27, 2022 at 09:33:27AM +0100, Daniel P. Berrangé wrote:
> > > On Tue, Sep 27, 2022 at 01:43:15PM +0530, Ani Sinha wrote:
> > > > On Sun, Sep 18, 2022 at 1:58 AM Michael S. Tsirkin <mst@redhat.com> 
> > > > wrote:
> > > > >
> > > > > On Fri, Sep 16, 2022 at 09:30:42PM +0530, Ani Sinha wrote:
> > > > > > On Thu, Jul 28, 2022 at 12:08 AM Ani Sinha <ani@anisinha.ca> wrote:
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On Mon, 25 Jul 2022, Ani Sinha wrote:
> > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On Sat, 16 Jul 2022, Michael S. Tsirkin wrote:
> > > > > > > >
> > > > > > > > > On Sat, Jul 16, 2022 at 12:06:00PM +0530, Ani Sinha wrote:
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On Fri, Jul 15, 2022 at 11:20 Michael S. Tsirkin 
> > > > > > > > > > <mst@redhat.com> wrote:
> > > > > > > > > >
> > > > > > > > > >     On Fri, Jul 15, 2022 at 09:47:27AM +0530, Ani Sinha 
> > > > > > > > > > wrote:
> > > > > > > > > >     > > Instead of all this mess, can't we just spawn e.g. 
> > > > > > > > > > "git clone --depth
> > > > > > > > > >     1"?
> > > > > > > > > >     > > And if the directory exists I would fetch and 
> > > > > > > > > > checkout.
> > > > > > > > > >     >
> > > > > > > > > >     > There are two reasons I can think of why I do not 
> > > > > > > > > > like this idea:
> > > > > > > > > >     >
> > > > > > > > > >     > (a) a git clone of a whole directory would download 
> > > > > > > > > > all versions of the
> > > > > > > > > >     > binary whereas we want only a specific version.
> > > > > > > > > >
> > > > > > > > > >     You mention shallow clone yourself, and I used --depth 
> > > > > > > > > > 1 above.
> > > > > > > > > >
> > > > > > > > > >     > Downloading a single file
> > > > > > > > > >     > by shallow cloning or creating a git archive is 
> > > > > > > > > > overkill IMHO when a wget
> > > > > > > > > >     > style retrieval works just fine.
> > > > > > > > > >
> > > > > > > > > >     However, it does not provide for versioning, tagging 
> > > > > > > > > > etc so you have
> > > > > > > > > >     to implement your own schema.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Hmm I’m not sure if we need all that. Bits has its own 
> > > > > > > > > > versioning mechanism and
> > > > > > > > > > I think all we need to do is maintain the same versioning 
> > > > > > > > > > logic and maintain
> > > > > > > > > > binaries of different  versions. Do we really need the 
> > > > > > > > > > power of git/version
> > > > > > > > > > control here? Dunno.
> > > > > > > > >
> > > > > > > > > Well we need some schema. Given we are not using official 
> > > > > > > > > bits releases
> > > > > > > > > I don't think we can reuse theirs.
> > > > > > > >
> > > > > > > > OK fine. Lets figuire out how to push bits somewhere in 
> > > > > > > > git.qemu.org and
> > > > > > > > the binaries in some other repo first. Everything else hinges 
> > > > > > > > on that. We
> > > > > > > > can fix the rest of the bits later incrementally.
> > > > > > >
> > > > > > > DanPB, any thoughts on putting bits on git.qemu.org or where and 
> > > > > > > how to
> > > > > > > keep the binaries?
> > > > > >
> > > > > > Can we please conclude on this?
> > > > > > Peter, can you please fork the repo? I have tried many times to 
> > > > > > reach
> > > > > > you on IRC but failed.
> > > > >
> > > > > Probably because of travel around KVM forum.
> > > > >
> > > > > I think given our CI is under pressure again due to gitlab free tier
> > > > > limits, tying binaries to CI isn't a great idea at this stage.
> > > > > Can Ani just upload binaies to qemu.org for now?
> > > >
> > > > I agree with Michael here. Having a full ci/cd job for this is
> > > > overkill IMHO. We should create a repo just for the binaries, have a
> > > > README there to explain how we generate them and check in new versions
> > > > as and when needed (it won't be frequent).
> > > > How about biosbits-bin repo?
> > >
> > > If QEMU is hosting binaries, where any part contains GPL code, then we
> > > need to be providing the full and corresponding source and the build
> > > scripts needed to re-create the binary. Once we have such scripts it
> > > should be trivial to trigger that from a CI job. If it isn't then
> > > we're doing something wrong.  The CI quota is not an issue, because
> > > this is not a job that we need to run continuously. It can be triggered
> > > manually as & when we decide we need to refresh the binary, so would
> > > be a small one-off CI quota hit.
> > >
> > > Also note that gitlab is intending to start enforcing storage quota
> > > on projects in the not too distant future. This makes it unappealing
> > > to store binaries in git repos, unless we genuinely need the ability
> > > to access historical versions of the binary. I don't believe we need
> > > that for biosbits.
> > >
> > > The binary can be published as a CI artifact and accessed directly
> > > from the latest artifact download link. This ensures we only consume
> > > quota for the most recently published binary artifact. So I don't see
> > > a compelling reason to upload binaries into git.
> >
> > I don't really care where we upload them but only having the
> > latest version is just going to break anything expecting
> > the old binary.
>
> biosbits isn't tied to QEMU versions, it is an entirely separate 3rd
> party project. This binary is just providing the test env, and IIUC,
> control over what executes in this env is still done by the QEMU side
> test scripts. I'm not seeing a coupling here that requires precise
> matching. In any case biosbit is a dead project so does not look
> likely to have any changes.
>
> If we did want to have different versions though, we can stil
> publish artifacts from different branches of biosbits code.

No, that is just ridiculous. Say we have a bug in bits that we fixed
and released a new version. Do we now create a new branch for that?
Multiple branches makes things needlessly complicated. We have one
branch, qemu-bits and all fixes go into that branch. We can have
different tags if we need. Nothing beyond that.

Gitlab
> will preserve & publish the latest artifacts from each branch in
> parallel.
>
> With regards,
> Daniel
> --
> |: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org         -o-            https://fstop138.berrange.com :|
> |: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
>