Re: Should we maybe move Cirrus-CI jobs away from Gitlab again?

[Daniel P . Berrangé]

On Tue, Sep 27, 2022 at 08:40:44PM +0200, Thomas Huth wrote:
> On 27/09/2022 19.57, Daniel P. Berrangé wrote:
> > On Tue, Sep 27, 2022 at 01:36:20PM -0400, Stefan Hajnoczi wrote:
> > > On Tue, 27 Sept 2022 at 11:54, Daniel P. Berrangé <berrange@redhat.com> 
> > > wrote:
> > > > 
> > > > On Tue, Sep 27, 2022 at 11:44:45AM -0400, Stefan Hajnoczi wrote:
> > > > > On Tue, 27 Sept 2022 at 05:02, Thomas Huth <thuth@redhat.com> wrote:
> > > > > > now that Gitlab is giving us pressure on the amount of free CI 
> > > > > > minutes, I
> > > > > > wonder whether we should maybe move the Cirrus-CI jobs out of the 
> > > > > > gitlab-CI
> > > > > > dashboard again? We could add the jobs to our .cirrus-ci.yml file 
> > > > > > instead,
> > > > > > like we did it in former times...
> > > > > > 
> > > > > > Big advantage would be of course that the time for those jobs would 
> > > > > > not
> > > > > > count in the Gitlab-CI minutes anymore. Disadvantage is of course 
> > > > > > that they
> > > > > > do not show up in the gitlab-CI dashboard anymore, so there is no 
> > > > > > more
> > > > > > e-mail notification about failed jobs, and you have to push to 
> > > > > > github, too,
> > > > > > and finally check the results manually on cirrus-ci.com ...
> > > > > 
> > > > > My understanding is that .gitlab-ci.d/cirrus.yml uses a GitLab CI job
> > > > > to run the cirrus-run container image that forwards jobs to Cirrus-CI.
> > > > > So GitLab CI resources are consumed waiting for Cirrus-CI to finish.
> > > > > 
> > > > > This shouldn't affect gitlab.com/qemu-project where there are private
> > > > > runners that do not consume GitLab CI minutes.
> > > > > 
> > > > > Individual developers are affected though because they most likely
> > > > > rely on the GitLab shared runner minutes quota.
> > > > 
> > > > NB, none of the jobs should ever be run automatically anymore in
> > > > QEMU CI pipelines. It always requires the maintainer to set the
> > > > env var when pushing to git, to explicitly create a pipeline.
> > > > You can then selectively start each individual job as desired.
> > > 
> > > Cirrus CI is not automatically started when pushing to a personal
> > > GitLab repo? If starting it requires manual action anyway then I think
> > > nothing needs to be changed here.
> > 
> > No pipeline at all is created unless you do
> > 
> >    git push -o ci.variable=QEMU_CI=1 <your-fork-remote>
> > 
> > that creates the pipeliune but doesn't run any jobs - they're manual
> > start.
> 
> Yes, sure, the jobs are not started automatically. But I *do* want to run
> the jobs before sending pull requests - but since the gitlab-CI minutes are
> now very limited, I'd like to avoid burning these minutes via gitlab and
> start those jobs directly on cirrus-ci.com again. For that the jobs would
> need to be moved to our .cirrus-ci.yml file again.

We do need a better story for maintainers sending pull requests to have
ability to run CI. We have 50+ jobs in the bujild stage of which the
cirrus jobs are just 3 - removing the cirrus jobs won't make a difference
to how quickly we run out of minutes if people try to run all of them.

We need to define a much tighter minimalist set of recommended jobs to
run.

I believe that if QEMU joins the OSS program, then the forks of QEMU
also benefit from a reduced cost factor for jobs they run, effectively
giving you much higher CI quota

> Well, maybe we could also have both, jobs via cirrus-run for those who want
> to see them in their gitlab-CI dashboard, and via .cirrus-ci.yml for those
> who want to avoid burning CI minutes on Gitlab. It's a little bit of
> double-maintenance, but maybe acceptable?

Key info about the jobs is in .gitlab-ci.d/cirrus/freebsd-12.vars which
could be referenced from the cirrus-ci.yml to reduce duplication

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|