Re: Azure infrastructure update

[Daniel P . Berrangé]

On Wed, Jun 28, 2023 at 01:41:03PM +0200, Paolo Bonzini wrote:
> > If we do this, then when opening a merge request, an initial pipeline
> > would be triggered.
> >
> > If-and-only-if the maintainer has "Developer" on gitlab.com/qemu-project,
> > then that merge request initial pipeline will burn upstream CI credits.
> >
> > If they are not a "Developer", it will burn their own fork credits. If
> > they don't have any credits left, then someone with "Developer" role
> > will have to spawn a pipeline on their behalf, which will run in
> > upstream context and burn upstream credits. The latter is tedious,
> > so I think expectation is that anyone who submits pull requests would
> > be expected to have 'Developer' role on qemu-project. We want that
> > anyway really so we can tag maintainers in issues on gitlab too.
> 
> Agreed. Is there no option to have the "Developer" use his own credits?

I've never found any way to control / force this behaviour.

> > IOW, assume that any maintainer opening a merge req will be burning
> > upstream CI credits on their merge request pipelines. [...]
> > Merge requests are not serialized though. [...]
> > To address this would require using GitLab's  "merge trains" feature.
> >
> > When merge trains are enabled, when someone hits the button to apply
> > a merge request to master, an *additional* CI pipeline is started
> > based on the exact content that will be applied to master. Crucially,
> > as the name hints, the merge train pipelines  are serialized. IOW,
> > if you request to apply 4 merge requests in quick succession a queue
> > of pipelines will be created and run one after the other. If any
> > pipeline fails, that MR is kicked out of the queue, and the
> > following pipelines carry on.
> >
> > IOW, the merge trains feature replicates what we achieve with the
> > serialized 'staging' branch.
> >
> > What you can see here though, is that every merge request will have
> > at least 2 pipelines - one when the MR is opened, and one when it
> > is applied to master - both consuming upstream CI credits.
> >
> > IOW, we potentially double our CI usage in this model if we don't
> > make any changes to how CI pipelines are triggered. [...]
> > If we can afford the CI credits, I'd keep things simple and
> > just accept the increased CI burn, but with your figures above
> > I fear we'd be too close to the limit to be relaxed about it.
> 
> Hmm, now that I think about it I'm not sure the merge request CI would
> use private runners. Would it use the CI variables that are set in
> settings/ci_cd? If not, the pipeline would not tag the jobs for
> private runners, and therefore the merge request would use shared
> runners (thus burning project minutes, but that's a different
> problem).

The repo/project global CI env variable settings should be
honoured for all pipelines in that repo, regardless of what
action triggers the pipeline. So I'd expect merge request
triggered pipelines to "just work" with the runner tagging.

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|