Getting build information in to the Guix Data Service (draft patch)

[Christopher Baines]

Hey,

I've now got a draft implementation of how Cuirass can be adapted to
push information about builds to the Guix Data Service, I'll send it
after this message.

I first attempted to get build information in to the Guix Data Service
back in April [1], but I only got as far as working out that trying to
request the status of every derivation, and then checking on derivations
that are still begin processed, but this approach didn't scale
particularly well and I only got as far as initially testing it.

1: https://lists.gnu.org/archive/html/guix-devel/2019-04/msg00094.html

In that email [1] I mentioned about trying to get the Guix Data Service
to somehow subscribe to Cuirass, so that when an event occurred (like a
build succeeding), the Guix Data Service would be informed.

I talked about WebSub as a pattern (and standard) for this, but recently
I realised that a good first step might be to try and get something
working without a hub in the middle, and just have Cuirass send events
directly to the Guix Data Service.

This is what I've currently tried to implement. The patch I'll send adds
two new tables to the Cuirass database, one to store events relating to
builds (like it being scheduled, or succeeding), and another to store
the ids of events which haven't yet been sent out. The code relating to
builds is then adjusted to populate these tables, and a new binary is
added to query for unsent events, and then send them out to some URL.

In the short term, the destination would be the Guix Data Service. In
the longer term, I think it would be better to send events to a WebSub
style hub, which then would distribute the events to one or more
subscribers.

Now that I've actually dug in to the Cuirass database to write this, I'm
more aware of the data model it uses, and the limitations this places on
what information it can provide. I'd assumed for a while that Cuirass
not showing complete information for each evaluation was a UI thing, but
as I understand it, the database only contains a record of what
derivations each evaluation has that no other evaluation in the
currently in the database has.

Maybe that's something to improve on within Cuirass, but at least by
getting the build information in to the Guix Data Service, it will be
feasible to look at the status of derivations for a revision of Guix (as
the Guix Data Service knows all the derivations associated with a
specific revision).

Do let me know what you think!

Thanks,

Chris

signature.asc
Description: PGP signature