Re: Multiple concurrent coprocesses

[Zachary Santer]

The attached script should demonstrate what I'm talking about, with
respect to commands buffering input.

Zack


On Mon, Mar 22, 2021 at 11:14 PM Zachary Santer <zsanter@gmail.com> wrote:
>
> > Something more than the trivial cases I've thrown at it. I'm interested in
> > whether or not the code has deadlock problems in the presence of pipelines.
>
> I've considered using sed with the --unbuffered flag in a coprocess, so I 
> could use a single sed invocation to handle arbitrary lines I feed it, as I 
> find them. In this case, the parent process would be echoing a line into the 
> coprocess and then immediately reading sed's output. Without sed's 
> --unbuffered flag, I can see how this could lead to a deadlock, as sed's 
> waiting for the parent process to fill its buffer while the parent process is 
> waiting for sed's response to the one line. If you're expecting one line back 
> for each line you feed into a coprocess containing a pipeline, you'd need 
> every command in the pipeline to have a similar option or just work in a 
> minimally-buffered manner out of the box. I don't know if anonymous pipes 
> themselves always wait to fill a buffer before sending their contents on. Am 
> I getting your meaning, though? I don't see how this would be more of a 
> problem with multiple coprocesses than it is with one.
>
> I feel it's up to the Bash programmer to recognize when something like the 
> above could happen, and find an alternative solution if it does. It wouldn't 
> be the fault of Bash itself if some commands buffer input or output no matter 
> what you do.
>
> In the case of my own script, there are things for the parent process and 
> both coprocesses to do at the same time; and buffering by the pipes 
> connecting the coprocesses to the parent process has the potential to 
> increase parallelism. All of this is still the case with the setup currently 
> in use, with FIFOs and automatic file descriptors, and I've not run into any 
> issues.
>
> Zack
>
> On Mon, Mar 22, 2021 at 9:33 AM Chet Ramey <chet.ramey@case.edu> wrote:
>>
>> On 3/21/21 4:38 PM, Zachary Santer wrote:
>> > Okay. I've at least seen job control report when each coprocess I created
>> > on the command line has ended, instead of waiting until they're all dead.
>> > And the coproc name going through name expansion is nice. Unless and until
>> > I'm allowed to bring my script home, I won't be able to show off what it's
>> > able to do, taking advantage of this Bash installation, though.
>> >
>> > I'm curious what satisfactory testing would look like.
>>
>> Something more than the trivial cases I've thrown at it. I'm interested in
>> whether or not the code has deadlock problems in the presence of pipelines.
>>
>> --
>> ``The lyf so short, the craft so long to lerne.'' - Chaucer
>>                  ``Ars longa, vita brevis'' - Hippocrates
>> Chet Ramey, UTech, CWRU    chet@case.edu    http://tiswww.cwru.edu/~chet/

sed-test
Description: Binary data