Re: Automatically building packages affected by submitted patches

[Christopher Baines]

Ricardo Wurmus <address@hidden> writes:

>> In short, following on from some previous emails about Patchwork [1] and
>> tracking and inspecting how Guix changes over time [2], I've got to the
>> point where I have a very rough setup for building packages changed by
>> patches sent to the guix-patches mailing list.
>
> This is great!

I'm glad you think so :)

>> With the goal of displaying a check in Patchwork about the
>> build status of the affected packages, I need to compare what's been
>> build by my Cuirass instance, with what https://ci.guix.info/ has
>> built. To do this, my current plan is to have the Guix Data Service
>> monitor a number of Cuirass instances somehow, extract information from
>> them and store it.
>
> Would it not make sense to have the build farm perform the builds
> instead of having a separate Cuirass instance?

I'm using my Cuirass instance because it's easy to test with.

But yes, I can see advantages in using the main build farm to perform
the builds, especially in potentially providing substitutes more
promptly. Were you thinking of anything in particular?

>>  - Looks at patch series in the Patchwork database that have been
>>    processed through the patchwork-test-series job in Laminar.
>
> Is the Laminar job something that could become part of Cuirass itself?

The patchwork-test-series job [1] running within Laminar is doing a few
things:

 - It takes the patches from patchwork.cbaines.net and applies them to
   the master branch.

 - (part 1) It pushes the resulting Git commits/branch up to [2].

 - (part 2) It instructs the Guix Data Service to fetch the base and tip
   of the branch, and load the data from those commits.

 - (part 3) It also stores the hashes of the commits in the same
   database used by Patchwork.

The above 3 parts of the process are a bit complicated, so I'll attempt
to explain how they fit together.

If you want to instruct Cuirass to build all the packages that are new
or that have been changed, you need to know what the state was before
the patches, and what the state with the patches looks like. This is
where part 2 comes in. Once the Guix Data Service has processed the two
commits, it can provide this information.

This is where part 1 comes in providing the patches in a more machine
readable form to the Guix Data Service and Cuirass, as they both fetch
from the Git branch created in part 1.

The Guix Data Service takes time to work out what's changed between the
two revisions. This is where storing the commit hashes against the
Patchwork series in the same database as Patchwork (part 3) comes
in. That allows regularly checking if there are any Patchwork series
that don't have a specification in Cuirass, and attempting to create one
using the information from the Guix Data Service.

1: https://laminar.cbaines.net/cfg/jobs/patchwork-test-series.run
2: https://git.cbaines.net/guix/patches

So back to your question, I don't think any of the above would fit
neatly within Cuirass. I'm still thinking of Cuirass as a build service,
you describe what you want it to build, and it goes away and does that.

One thing I've become aware of recently is how similar Cuirass and the
Guix Data Service actually are. The evaluations part of Cuirass and
storing the derivations is very similar to the Guix Data Service.

Do let me know if you have more thoughts or questions...

Chris

signature.asc
Description: PGP signature