[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Serious doubts about Gitlab CI
From: |
Philippe Mathieu-Daudé |
Subject: |
Re: Serious doubts about Gitlab CI |
Date: |
Tue, 30 Mar 2021 14:33:02 +0200 |
User-agent: |
Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.8.0 |
On 3/30/21 1:55 PM, Thomas Huth wrote:
> On 30/03/2021 13.19, Daniel P. Berrangé wrote:
>> On Mon, Mar 29, 2021 at 03:10:36PM +0100, Stefan Hajnoczi wrote:
>>> Traditionally ccache (https://ccache.dev/) was used to detect
>>> recompilation of the same compiler input files. This is trickier to do
>>> in GitLab CI since it would be necessary to share and update a cache,
>>> potentially between untrusted users. Unfortunately this shifts the
>>> bottleneck from CPU to network in a CI-as-a-Service environment since
>>> the cached build output needs to be accessed by the linker on the CI
>>> runner but is stored remotely.
>>
>> Our docker containers install ccache already and I could have sworn
>> that we use that in gitlab, but now I'm not so sure. We're only
>> saving the "build/" directory as an artifact between jobs, and I'm
>> not sure that directory holds the ccache cache.
>
> AFAIK we never really enabled ccache in the gitlab-CI, only in Travis.
Back then the Travis setup was simpler, and it took me 2 to 3 weeks
to get it right (probably spending 3 to 4h a day on it).
>>> This is as far as I've gotten with thinking about CI efficiency. Do you
>>> think these optimizations are worth investigating or should we keep it
>>> simple and just disable many builds by default?
>>
>> ccache is a no-brainer and assuming it isn't already working with
>> our gitlab jobs, we must fix that asap.
>
> I've found some nice instructions here:
>
> https://gould.cx/ted/blog/2017/06/10/ccache-for-Gitlab-CI/
>
> ... and just kicked off a build with these modifications, let's see how
> it goes...
But we cross-build in Docker containers, so you need to mount the
cache dir in the container and set the CCACHE_DIR env var, isn't it?
Watch out about custom runners. If we do too many changes on the
free-tier runners, we'll never have the custom runner series integrated.
My 2 cents.
Regards,
Phil.
- Re: Serious doubts about Gitlab CI, (continued)
- Re: Serious doubts about Gitlab CI, Daniel P . Berrangé, 2021/03/30
- Re: Serious doubts about Gitlab CI, Paolo Bonzini, 2021/03/30
- Re: Serious doubts about Gitlab CI, Daniel P . Berrangé, 2021/03/30
- Re: Serious doubts about Gitlab CI, Warner Losh, 2021/03/30
- Re: Serious doubts about Gitlab CI, Peter Maydell, 2021/03/30
- Re: Serious doubts about Gitlab CI, Warner Losh, 2021/03/30
- Re: Serious doubts about Gitlab CI, Thomas Huth, 2021/03/30
- Re: Serious doubts about Gitlab CI, Daniel P . Berrangé, 2021/03/30
- Re: Serious doubts about Gitlab CI, Paolo Bonzini, 2021/03/30
- Re: Serious doubts about Gitlab CI, Peter Maydell, 2021/03/30
- Re: Serious doubts about Gitlab CI,
Philippe Mathieu-Daudé <=
- Re: Serious doubts about Gitlab CI, Daniel P . Berrangé, 2021/03/30
- Re: Serious doubts about Gitlab CI, Thomas Huth, 2021/03/31
- Re: Serious doubts about Gitlab CI, Andrea Bolognani, 2021/03/31
- Re: Serious doubts about Gitlab CI, Stefan Hajnoczi, 2021/03/30
- Re: Serious doubts about Gitlab CI, Paolo Bonzini, 2021/03/30
- Re: Serious doubts about Gitlab CI, Daniel P . Berrangé, 2021/03/30
- Re: Serious doubts about Gitlab CI, Stefan Hajnoczi, 2021/03/30
- Re: Serious doubts about Gitlab CI, Markus Armbruster, 2021/03/19
- Re: Serious doubts about Gitlab CI, Thomas Huth, 2021/03/19