Re: Examples of concurrent coproc usage?

[Chet Ramey]

On 4/9/24 11:58 AM, Carl Edquist wrote:
> On 4/4/24 7:23 PM, Martin D Kealey wrote:
>
> > I'm somewhat uneasy about having coprocs inaccessible to each other. I 
> > can foresee reasonable cases where I'd want a coproc to utilize one or 
> > more other coprocs.
> >
> > In particular, I can see cases where a coproc is written to by one 
> > process, and read from by another.
> >
> > Can we at least have the auto-close behaviour be made optional, so that 
> > it can be turned off when we want to do something more sophisticated?
>
> With support for multiple coprocs, auto-closing the fds to other coprocs 
> when creating new ones is important in order to avoid deadlocks.
>
> But if you're willing to take on management of those coproc fds yourself, 
> you can expose them to new coprocs by making your own copies with exec 
> redirections.
>
> But this only "kind of" works, because for some reason bash seems to close 
> all pipe fds for external commands in coprocs, even the ones that the user 
> explicitly copies with exec redirections.



> (More on that in a bit.)
>
>
> On Mon, 8 Apr 2024, Chet Ramey wrote:
>
> > On 4/4/24 7:23 PM, Martin D Kealey wrote:
> > > I'm somewhat uneasy about having coprocs inaccessible to each other. I 
> > > can foresee reasonable cases where I'd want a coproc to utilize one or 
> > > more other coprocs.

> > That's not the intended purpose,

The original intent was to allow the shell to drive a long-running process
that ran more-or-less in parallel with it. Look at examples/scripts/bcalc
for an example of that kind of use.

> For what it's worth, my experience is that coprocesses in bash (rigged up 
> by means other than the coproc keyword) become very fun and interesting 
> when you allow for the possibility of communication between coprocesses. 
> (Most of my use cases for coprocesses fall under this category, actually.)

Sure, as long as you're willing to take on file descriptor management
yourself. I just don't want to make it a new requirement, since it's never
been one before.

> Now, if you built bash with multiple coproc support, I would have expected 
> you could still rig this up, by doing the redirection work explicitly 
> yourself.  Something like this:
>
>      coproc UP   { stdbuf -oL tr a-z A-Z; }
>      coproc DOWN { stdbuf -oL tr A-Z a-z; }
>
>      # make user-managed backup copies of coproc fds
>      exec {up_r}<&${UP[0]} {up_w}>&${UP[1]}
>      exec {down_r}<&${DOWN[0]} {down_w}>&${DOWN[1]}
>
>      coproc THREEWAY { tee /dev/fd/$up_w  /dev/fd/$down_w; }
>
>
> But the above doesn't actually work, as it seems that the coproc shell 
> (THREEWAY) closes specifically all the pipe fds (beyond 0,1,2), even the 
> user-managed ones explicitly copied with exec.

File descriptors the user saves with exec redirections beyond [0-2]
are set to close-on-exec. POSIX makes that behavior unspecified, but
bash has always done it.

Shells don't offer any standard way to modify the state of that flag,
but there is the `fdflags' loadable builtin you can experiment with
to change close-on-exec.

Chet

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
                 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU    chet@case.edu    http://tiswww.cwru.edu/~chet/

OpenPGP_signature.asc
Description: OpenPGP digital signature