Re: The Shepherd on Fibers

[Zhu Zihao]

Philip McGrath <philip@philipmcgrath.com> writes:

> Would it be feasible for shepherd *not* to fork? Or only to fork in a way that
> cooperates with fibers?

IMO the problem is not fork, you can't do anything useful to create
subprocess without fork on *NIX systems.

fork is allowed to execute in multi-thread environment. It just don't
care about other threads, it's safe and recommended if you only call
execv after fork.

The problem is, Shepherd is too flexible and people can do so many
things other than fork+exec in the startup of Shepherd service (unlike
systemd, only create subprocess is allowed). So Shepherd has to be
conservative.

> Second, I'm a little uneasy about `unwind-protect`:
>
> ```
> +(define-syntax-rule (unwind-protect body ... conclude)
> +  "Evaluate BODY... and return its result(s), but always evaluate CONCLUDE
> +before leaving, even if an exception is raised.
> +
> +This is *not* implemented with 'dynamic-wind' in order to play well with
> +delimited continuations and fibers."
> +  (let ((conclusion (lambda () conclude)))
> +    (catch #t
> +      (lambda ()
> +        (call-with-values
> +            (lambda ()
> +              body ...)
> +          (lambda results
> +            (conclusion)
> +            (apply values results))))
> +      (lambda args
> +        (conclusion)
> +        (apply throw args)))))
> ```
>
> though maybe it's used only internally and doesn't have to deal with the 
> general
> case: I'm not an expert by any means, but my understanding (from a Racket
> mailing list thread at [5], continued at [6]) is that dealing with the general
> case may be something of an open research question.
>
> As an example of the sort of problem, what if the `body`s evaluate normally, 
> but
> an exception is raised in the dynamic extent of `(conclusion)`, causing
> `(conclusion)` to run again? One possible mitigation would be to `set!` a 
> local
> variable before the normal `(conclusion)` and check it in the exception 
> handler.
>
> More generally, of course, `body ...` could escape by some mechanism other 
> than
> a raising an exception.
>
> If you were writing Racket, I would recommend `(call-with-continuation-barrier
> (λ () body ...))`—logically, `body ...` isn't re-entrant anyway—but I see from
> [7] that Guile's continuation barriers are barriers to the fibers' context
> switches.

> The key difference between Guile and Racket is that Guile's Fibers are just a
> library, without any special privileges, and use the same control operators 
> that
> are available to user code. I think that may be unique among the Concurrent ML
> implementations I'm aware of.
>
> Why is that a problem? Well, in many ways I think it's quite cool! But a
> problematic aspect is that (from the discussion at [5]):
>
>
>> On 11/30/19 10:23, Matthew Flatt wrote:
>>> You're trying to implement `amb` where a client can mix `amb` and
>>> `dynamic-wind` and get sensible behavior, right?
>>> The `dynamic-wind` operation certainly interferes with building new
>>> control forms. Racket threads and other control forms that need to
>>> interact a certain way with `dynamic-wind` end up being built in at the
>>> same level as `dynamic-wind`.
>>> At Sat, 30 Nov 2019 06:15:16 -0600, Alexis King wrote:
>>>> Is there any way to do that with Racket’s continuation machinery,
>>> There's not a safe way. In many cases, Racket lets you write new things
>>> that have the power of built-in through unsafe APIs --- and it turns
>>> out that there are unadvertised procedures (provided by the primitive
>>> `#%unsafe` module) for this particular case:
>>> [...]
>>> At Sat, 30 Nov 2019 06:15:16 -0600, Alexis King wrote:
>>>> Also, is this kind of thing discussed anywhere in the literature?
>>> I don't know of any published work on this topic (so let me know if you
>>> find something!). As you probably have seen already, our ICFP'07 paper
>>> just points out that `dynamic-wind` causes problems, but doesn't try to
>>> hold `dynamic-wind` itself responsible for those problems.
>>> An opinion and some pointers to newsgroup discussions:
>>>    http://okmij.org/ftp/continuations/against-callcc.html#dynamic_wind
>>>

The reentrant feature of call/cc costs many but worth nothing. And people
have to create dynamic-wind to fix this ugly issue.

Schemers now create a new proposal, SRFI-226. In this proposal, call/cc
is reimplemented in delimited control operator.

> Maybe it would be enough for this case for Fibers to provide variants of
> `dynamic-wind` and/or `call-with-continuation-barrier` that cooperate 
> with the Fibers implementation. I don't know if that would be possible of 
> not—in
> addition to my limited familiarity with Fibers, I have not read all of the
> related literature that Alexis, Matthew, and Matthias Felleisen discuss in [5]
> and [6]—again, I am not an expert!
>
> I'm not sure how useful any of that is, but take it for whatever it's worth. 
> My
> overall impression is that the Shepherd on Fibers is a big step in the right
> direction. Congrats on the great work!

If Shepherd can use something like G-exp, we can force user to do its
customized job in subprocess via code staging. And restrict the shepherd
only do process creation in the startup of service. 
-- 
Retrieve my PGP public key:

  gpg --recv-keys D47A9C8B2AE3905B563D9135BE42B352A9F6821F

Zihao

signature.asc
Description: PGP signature