[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [PATCH RFC 0/3] gitlab: build containers to use in build jobs
From: |
Daniel P . Berrangé |
Subject: |
Re: [PATCH RFC 0/3] gitlab: build containers to use in build jobs |
Date: |
Thu, 25 Jun 2020 12:26:53 +0100 |
User-agent: |
Mutt/1.14.0 (2020-05-02) |
On Thu, Jun 25, 2020 at 01:15:52PM +0200, Thomas Huth wrote:
> On 22/06/2020 17.33, Daniel P. Berrangé wrote:
> > The current gitlab CI jobs are quite inefficient because they
> > use the generic distro images and then apt-get/dnf install
> > extra packages every time.
> >
> > The other downside is that the container environment used is
> > only defined in thte .gitlab-ci.yml file, so it tedious to
> > reproduce locally.
> >
> > We already have containers defined in tests/docker for use by
> > developers building locally. We can use these for CI systems
> > too if we just had a way to build them....
> >
> > ...GitLab CI offers such a way. We can use docker-in-docker
> > to build the images at the start of the CI cycle, and use
> > the built images in later jobs.
> >
> > These later jobs are now faster because they're not having
> > to install any software.
>
> Did you see any speed-up? I had a look at some pipelines, and it seems to me
> that they rather got slower now? For example, this is the system1 pipeline
> before your change:
>
> https://gitlab.com/huth/qemu/-/jobs/610924897
>
> and after your change:
>
> https://gitlab.com/huth/qemu/-/jobs/611069374
>
> Duration went up from 35 minutes to 42 minutes.
>
> Seems also to happen in your builds, before the change:
>
> https://gitlab.com/berrange/qemu/-/jobs/582995084
>
> and after the change:
>
> https://gitlab.com/berrange/qemu/-/jobs/606175927
>
> ... went from 36 minutes up to 42 minutes.
>
> Could be a coincidence due to the load on the shared runners, but it looks
> at least a little bit suspicious...
I think the difference is because we're building more features now. The
dockerfiles have provided more build pre-requisites that the old gitlab
recipe did.
If you compare the configure summary, I see the new build now covers
SDL, curses, curl, pulseaudio, virtiofs, SASL, libjpeg, xen, docs
and a few more. So we've saved time by not intsallling many packages
each time, but consumed a greater amount of time by compiling more
features.
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
- Re: [PATCH RFC 2/3] gitlab: build all container images during CI, (continued)
Re: [PATCH RFC 2/3] gitlab: build all container images during CI, Thomas Huth, 2020/06/25
Re: [PATCH RFC 2/3] gitlab: build all container images during CI, Daniel P . Berrangé, 2020/06/25
Re: [PATCH RFC 2/3] gitlab: build all container images during CI, Thomas Huth, 2020/06/25
Re: [PATCH RFC 0/3] gitlab: build containers to use in build jobs, Alex Bennée, 2020/06/25
Re: [PATCH RFC 0/3] gitlab: build containers to use in build jobs, Thomas Huth, 2020/06/25
- Re: [PATCH RFC 0/3] gitlab: build containers to use in build jobs,
Daniel P . Berrangé <=