Re: make calling script calling make results in read jobs pipe: Bad file descriptor. Stop.

[Martin d'Anjou]

Thanks Paul, but I am confused by this paragraph:

>However, make ALWAYS sends along the MAKEFLAGS variable to all commands
>it runs, whether they are sub-makes or not.  And, there is a bug (filed
>in Savannah) that even when make determines that the process is not a
>sub-make, it will still send along the jobserver information in
>MAKEFLAGS.

The confusing parts to me: "make ALWAYS sends along the MAKEFLAGS" and "it
will still send along the jobserver information in MAKEFLAGS".

If make already always sends MAKEFLAGS, why say that the bug fix
proposes to also always send MAKEFLAGS. I am not getting it... it's already
doing it.

Martin





On Thu, Mar 4, 2021 at 1:57 PM Paul Smith <psmith@gnu.org> wrote:

> On Thu, 2021-03-04 at 12:24 -0500, Martin d'Anjou wrote:
> > The process tree is like this:
> > gnumake(1) -> cmdA(2) -> cmdB(3) -> cmdC(4)-> ... -> gnumake(7)
> >
> > The first make is called with the -j option. The intermediate
> > commands are not controlled by me, they are proprietary vendor code
> > that I cannot access. The last call to make is also hidden from me.
>
> > Help me understand what is happening. Is it a limitation/bug of GNU
> > Make, an unsupportable use case, what is happening? Any suggestions
> > on how to fix this?
>
> There are a number of recent issues filed around this issue.
>
> What happens is that the top-level make is started with -jN so it
> creates the jobserver infrastructure, which is a pipe.  make also puts
> the file descriptors of the pipe into the MAKEFLAGS variable so that
> sub-makes can find it.
>
> When make invokes a command in a recipe, if it thinks that the command
> is a sub-make it will leave the pipes open so that the sub-make can
> participate in the jobserver.
>
> If the top-level make believes that the command is NOT a sub-make, it
> closes the pipes so that the sub-command won't get confused about these
> extra open file descriptors.
>
> I assume that in your situation, make is determining that cmdA is not a
> sub-make and so is closing the pipes.
>
> However, make ALWAYS sends along the MAKEFLAGS variable to all commands
> it runs, whether they are sub-makes or not.  And, there is a bug (filed
> in Savannah) that even when make determines that the process is not a
> sub-make, it will still send along the jobserver information in
> MAKEFLAGS.
>
> This normally is not a problem.  Process that are not make obviously
> don't care about MAKEFLAGS.  If the sub-process IS a make and sees
> MAKEFLAGS it will try to use the jobserver pipes, see they are closed,
> and notify the user that jobserver is not available.
>
> However, there are problems in situations like this:
> 1) Make closes jobserver descriptors and invokes command
> 2) Command opens something using the same descriptor
> 3) Command itself runs make, leaving those descriptors open
> 4) make tries to use those descriptors as jobserver descriptors
>
> To fix this, in a future version of make will will do at least one, and
> maybe both, of two things:
> A) when we close the file descriptors, update MAKEFLAGS to remove them
> so that sub-makes won't try to use them.
> B) switch away from simple pipes to using named pipes for jobserver
> implementation; then we don't need to worry about leaving descriptors
> open
>
> To fix the issue for now in your environment, you'll need to perform
> the equivalent of solution (A) in your own environment.
>
> That is, whatever recipe you have that invokes cmdA will need to clean
> up MAKEFLAGS; either something simple like:
>
>   run-cmdA:
>           MAKEFLAGS= cmdA
>
> Or, if you prefer, you can try to just strip out the flags you don't
> want, something like this:
>
>   run-cmdA:
>           MAKEFLAGS='$(filter-out --jobserver-auth%,$(MAKEFLAGS))' cmdA
>
> Obviously the details are up to you.
>
>