Re: [PULL 00/17] Net patches

[Ilya Maximets]

On 9/19/23 10:40, Daniel P. Berrangé wrote:
> On Mon, Sep 18, 2023 at 09:36:10PM +0200, Ilya Maximets wrote:
>> On 9/14/23 10:13, Daniel P. Berrangé wrote:
>>> On Wed, Sep 13, 2023 at 08:46:42PM +0200, Ilya Maximets wrote:
>>>> On 9/8/23 16:15, Daniel P. Berrangé wrote:
>>>>> On Fri, Sep 08, 2023 at 04:06:35PM +0200, Ilya Maximets wrote:
>>>>>> On 9/8/23 14:15, Daniel P. Berrangé wrote:
>>>>>>> On Fri, Sep 08, 2023 at 02:00:47PM +0200, Ilya Maximets wrote:
>>>>>>>> On 9/8/23 13:49, Daniel P. Berrangé wrote:
>>>>>>>>> On Fri, Sep 08, 2023 at 01:34:54PM +0200, Ilya Maximets wrote:
>>>>>>>>>> On 9/8/23 13:19, Stefan Hajnoczi wrote:
>>>>>>>>>>> Hi Ilya and Jason,
>>>>>>>>>>> There is a CI failure related to a missing Debian libxdp-dev 
>>>>>>>>>>> package:
>>>>>>>>>>> https://gitlab.com/qemu-project/qemu/-/jobs/5046139967
>>>>>>>>>>>
>>>>>>>>>>> I think the issue is that the debian-amd64 container image that QEMU
>>>>>>>>>>> uses for testing is based on Debian 11 ("bullseye" aka "oldstable")
>>>>>>>>>>> and libxdp is not available on that release:
>>>>>>>>>>> https://packages.debian.org/search?keywords=libxdp&searchon=names&suite=oldstable&section=all
>>>>>>>>>>
>>>>>>>>>> Hmm.  Sorry about that.
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> If we need to support Debian 11 CI then either XDP could be disabled
>>>>>>>>>>> for that distro or libxdp could be compiled from source.
>>>>>>>>>>
>>>>>>>>>> I'd suggest we just remove the attempt to install the package for 
>>>>>>>>>> now,
>>>>>>>>>> building libxdp from sources may be a little painful to maintain.
>>>>>>>>>>
>>>>>>>>>> Can be re-added later once distributions with libxdp 1.4+ will be 
>>>>>>>>>> more
>>>>>>>>>> widely available, i.e. when fedora dockerfile will be updated to 39,
>>>>>>>>>> for example.  That should be soon-ish, right?
>>>>>>>>>
>>>>>>>>> If you follow the process in docs/devel/testing.rst for adding
>>>>>>>>> libxdp in libvirt-ci, then lcitool will "do the right thing"
>>>>>>>>> when we move the auto-generated dockerfiles to new distro versions.
>>>>>>>>
>>>>>>>> Thanks!  I'll prepare changes for libvirt-ci.
>>>>>>>>
>>>>>>>> In the meantime, none of the currently tested images will have a 
>>>>>>>> required
>>>>>>>> version of libxdp anyway, so I'm suggesting to just drop this one 
>>>>>>>> dockerfile
>>>>>>>> modification from the patch.  What do you think?
>>>>>>>
>>>>>>> Sure, if none of the distros have it, then lcitool won't emit the
>>>>>>> dockerfile changes until we update the inherited distro version.
>>>>>>> So it is sufficient to just update libvirt-ci.git with the mappings.yml
>>>>>>> info for libxdp, and add 'libxdp' to the tests/lcitool/projects/qemu.yml
>>>>>>> file in qemu.git. It will then 'just work' when someone updates the
>>>>>>> distro versions later.
>>>>>>
>>>>>> I posted an MR for libvirt-ci adding libxdp:
>>>>>>   https://gitlab.com/libvirt/libvirt-ci/-/merge_requests/429
>>>>>>
>>>>>> Please, take a look.
>>>>>>
>>>>>> The docs say that CI will try to build containers with the MR changes,
>>>>>> but I don't think anything except sanity checks is actually tested on MR.
>>>>>> Sorry if I missed something, never used GitLab pipelines before.
>>>>>
>>>>> No, that's our fault - we've broken the CI and your change alerted
>>>>> me to that fact :-)
>>>>>
>>>>>> Note that with this update we will be installing older version of libxdp
>>>>>> in many containers, even though they will not be used by QEMU, unless
>>>>>> they are newer than 1.4.0.
>>>>>
>>>>> No problem, as it means QEMU CI will demonstrate the the meson.build
>>>>> change is ignoring the outdatd libxdp.
>>>>>
>>>>>> tests/lcitool/projects/qemu.yml in qemu.git cannot be updated without
>>>>>> updating a submodule after the MR merge.
>>>>>
>>>>> Yep.
>>>>
>>>> Since all the required changes went into libvirt-ci project, I posted an
>>>> updated patch set named:
>>>>
>>>>   '[PATCH v4 0/2] net: add initial support for AF_XDP network backend'
>>>>
>>>> Please, take a look.
>>>>
>>>> This should fix the CI issues, though I'm not sure how to run QEMU gitlab
>>>> pipelines myself, so I didn't actually test all the images.
>>>
>>>   git push gitlab  -o ci.variable=QEMU_CI=2
>>>
>>> will create pipeline and immediately run all jobs.
>>
>> Thanks!  That worked.  Though I wasn't able to test much anyway as
>> this thing burned through all my free compute credits less than
>> half way through the pipeline. :D
>>
>> So, AFAIU, it's not something an occasional contributor like me can
>> use, unless they are spending their own money.
> 
> That is not the expected behaviour.
> 
> If your repo is a fork of https://gitlab.com/qemu-project/qemu it
> should benefit from a *massive* x125 reduction on CI costs.
> 
> The critical thing is that it *MUST* have been created with the
> 'Fork' button on qemu-project/qemu.

Yeah, it might be that the problem is caused by me accidentally
forking the gitlab.com/qemu/qemu repo instead of qemu-project.

It is fairly confusing that qemu/qemu is not the main repository
of QEMU project.  It seems to be some sort of automated account
and it closely follows updates of the main repository.  It also
has a better name, and it is *not a fork* of the qemu-project.
There practically no indication that qemu/qemu is not a main
repository, except for an icon and a lower star/fork count.
It's easy to fork the wrong one.

Do you folks have control over this account?  Could you maybe add
a description that it is not the official QEMU repository and add
a link to qemu-project?  Right now qemu-project/qemu states that
it is a "QEMU main repository", but qemu/qemu doesn't say anything.


> If that's not the case then
> you will burn CI credits at a cost of 1 minute == 1 credit,
> instead of 1 minute == 0.008 credits. Check this by going to
> the top page of your repo, and looking for a box a little above
> the file list, that says
> 
>     "Forked from QEMU / QEMU"
> 
> If that is not the case, then you'll have to rename your existing
> repo to get it out of the way, and then use the 'Fork' button to
> create a new copy that is tracked as a fork.
> 
> With most accounts getting 400 CI minutes per month, an averge
> QEMU CI run should consume about 7 CI minutes.
> 
> NB, CI credits reset on the 1st of each month

I deleted my fork (there wasn't anything useful there) and re-forked
the correct one.  Will try again next month.  For now "No more CI
minutes available.  You have used 788 out of 400 of your shared Runners
pipeline minutes." :D

Best regards, Ilya Maximets.