Re: Getting build information in to the Guix Data Service (draft patch)

[Christopher Baines]

Christopher Baines <address@hidden> writes:

> Ludovic Courtès <address@hidden> writes:
>
>> Christopher Baines <address@hidden> skribis:
>>
>> Should ‘BuildEvents’ be more generic and have ‘event’ be an sexp or JSON
>> string that could describe any kind of event?
>>
>> If we did that, we could keep ‘derivation’ but remove “NOT NULL” so that
>> non-derivation events can exist but we can still query
>> derivation-related events quickly.  Does that make sense?
>
> Yep, that makes sense.
>
> This seems to be the general decision about the way you use a relational
> database, do you have specific tables (types) for the data, or do you
> have a more freeform structure (columns containing sexp or JSON).
>
> There's quite a few factors to consider here, the internals of Cuirass,
> how these events are exposed through the HTTP API, how these events one
> day might be published to a WebSub hub and then what kind of
> subscriptions you might support in Cuirass (events for an individual
> derivation, all builds for an evaluation, all builds, ...). I'll think
> about it further and see if I can form an opinion either way.

So I had a think about this, and also re-read the WebSub spec [1], along
with JSON Feed which is a Atom/RSS alternative with JSON [2]. I'm still
a bit undecided about how closely to follow WebSub, as it seems more
concerned with applications like blogging, and less with moving data
between services, but I still think the principles are good. For now, I
think it's best to pick the bits of the standard that work, and leave
any that don't particularly apply.

I think a generic approach to storing the events in the database will
work better though, at least with the way the events are currently
exposed. It does make it potentially harder to expose and support
subscriptions for events for individual builds, but that can probably be
addressed later if desired.

1: https://www.w3.org/TR/websub/
2: https://jsonfeed.org/version/1

>>> +CREATE TABLE BuildEventsOutbox (
>>> +  build_event_id INTEGER NOT NULL,
>>> +  FOREIGN KEY (build_event_id) REFERENCES BuildEvents (id)
>>> +);
>>
>> These are events that have not yet been sent, right?
>
> Yep, exactly.
>
>> Thanks!
>
> Thanks for taking a look. I'll neaten up the patch a bit, add in some
> error handling and retrying for sending out the events, and think a bit
> more about the data model, then hopefully send an updated patch soon!

I've now sent an updated set of patches. The first to add events for
builds, and the second to extend this to evaluations.

I've also added in error handling to the sending of the events.

Thanks,

Chris

signature.asc
Description: PGP signature