[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Ungrafting CI jobs
From: |
Mathieu Othacehe |
Subject: |
Re: Ungrafting CI jobs |
Date: |
Wed, 02 Jun 2021 19:53:35 +0200 |
User-agent: |
Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) |
Hello Leo,
I monitored a previous evaluation of the ungrafting Cuirass
specification that was more successful, with more than 17000 builds
performed in less than 24 hours, a new record! The recent evaluation is
sadly less victorious.
> The latest iteration failed en masse due to network timeouts between
> build farm nodes.
Yes, I described this issue here: https://issues.guix.gnu.org/48468 and
hopefully fixed it with https://issues.guix.gnu.org/48556.
I need to perform a reconfigure / deploy but I'm waiting to have a few
days ahead of me to fix potential issues.
> It lets us see that the Audacity build is not attempted due to the
> failure of wxwidgets. And in turn, the failure of webkitgtk and libsoup.
> However, there is no information (logs, duration) available about those
> failed builds.
When a build is cancelled ("Cancel pending builds" button), all
its dependents are automatically updated to "Failed dependency". It
looks like this is the case for https://ci.guix.gnu.org/build/515005/details.
> What can we do to clear all these "spurious" build failures and re-try
> building the branch? I already tried using the "restart all builds"
> button, but there were still a lot of spurious failures.
I think the easiest way to recover from this situation is to remove the
ungrafting specification and recreate it. It ensures that no build will
fail because of previously failing/cancelled build dependencies.
Note that when a build B is restarted in Cuirass ("Restart" button), any
other build B' depending from B, will automatically be restarted if B
becomes successful and all the other dependencies of B' are already
successful.
I used this mechanism to restart a few builds failing on the master
branch because of substitutes timeout. I hope that when the keep-alive
patchset will solve this situation.
Thanks,
Mathieu