Re: (Exposing?) config files and non-start/stop operations

[Christopher Allan Webber]

Chris Marusich writes:

>>>  - Initializing a data store.  For example, in dirvish I need to run
>>>    a command to initialize a "vault" where I will be storing my data.
>>>  - Manually invoking a garbage collection utility.
>>>  - Manually invoking an integrity check utility.
>>>  - Possibly some side effect involving querying the network.
>>>  - Running schema migrations.
>>>
>>> All sorts of things!  Most of them (all?) involve state or side effects,
>>> but plenty of our most important services will be "state overlords" of
>>> some type.
>
> Why do those activities need to be represented as actions in Shepherd?
> If we're running a daemon or service that already exposes a mechanism
> for manually running tasks like these, then can't we just use "the usual
> mechanism" for doing it?  For example, if we're running a daemon that
> already provides a script to perform garbage collection, can't we just
> invoke that script?  It isn't clear to me why we would need to model
> domain-specific actions like that in Shepherd, although I can see why it
> might be convenient.  Am I missing something?

So sure, we can run "foo-db gc" occasionally (though system
administrators sometimes have to run these kinds of commands by hand).
But what about "foo-db dumpdb"?  That's not something we just run on a
cronjob.  You need access to that command.  And in order for the command
to do the right thing, it might need access to the config file.

There are some other things where we can try to initialize or run
migrations automatically, but that stuff can be very dangerous to just
do implicitly.  Now note, I *do* think we want to handle some of this
stuff behind the scenes as much as possible so that users don't have to
think about it.  But have you ever done a *really big* database schema
migration?

We run into two challenges:
 - Now we're trying to "idempotently manage state", which it turns out
   is very hard (and the source of many bugs in devops tooling)
 - Some commands either need to be run manually occasionally, or are
   never automatically run (see the dumpdb example).

Does this make sense?
 - Chris