Re: GNU make jobserver redux

[Paul D. Smith]

%% Paul Eggert <address@hidden> writes:

  >> From: "Paul D. Smith" <address@hidden>
  >> Date: Fri, 4 May 2001 01:58:24 -0400
  >> 
  >> The trick is we need the read() to be interruptible on signals,
  >> otherwise we'll never reap any children until we get a token... which
  >> could deadlock us.

  pe> Won't SA_RESTART let you implement that trick without putting EINTR
  pe> checks all over the place?

  pe> SA_RESTART lets the read() be interruptible, but it causes an
  pe> interrupted read() to be restarted after the signal handler returns.
  pe> Therefore, read() never fails with errno==EINTR; it either succeeds,
  pe> or it fails with errno!=EINTR (typically errno==EBADF).  This is the
  pe> behavior that you want.

Yes, but I don't think this will work.

We discussed and discarded it last time (I went through all my old
archives of this discussion before posting my message, to be sure I
remembered all the issues).

  pe> Perhaps this SA_RESTART idea wasn't used before, either because the
  pe> SA_RESTART feature wasn't available long ago when you first worked on
  pe> this, or because some older Unix systems didn't have SA_RESTART.
  pe> However, SA_RESTART support is becoming standard.  It's supported on
  pe> modern versions of GNU/Linux, BSD, and Solaris (the systems I have
  pe> easy access to).  POSIX 1003.1-200x Draft 6 specifies SA_RESTART, so
  pe> it's becoming the consensus standard.  I think it's OK for "make" to
  pe> not support the jobserver feature on older systems that lack
  pe> SA_RESTART, given that SA_RESTART makes the code much simpler and more
  pe> reliable.

No, it's not the SA_RESTART portability that's the problem.  Already we
only enable jobserver on the hosts which have sufficient capability to
handle it, and that's OK with me.  Make works serially on the other
systems, you don't have to write your makefiles differently to use them
serially, so that's portable enough given that most systems do support
SA_RESTART.

The problem, IIRC, is that we concluded that some, most, or all systems
_don't_ return with EBADF if you close the file descriptor in the
handler.

That is, once you start the read() with a valid FD, even if the signal
handler closes that FD then the read() still continues and does not
return with EBADF.

If the read() doesn't return, then our algorithm has failed utterly.

It's _certainly_ the case that this happens on Hurd, because there FDs
are simply handles for underlying kernel structures, which won't go away
as long as they're in use (if I recall Roland's explanation correctly
after all this time).

I believe I wrote a test program for this and we passed it around via
email, and concluded that some fairly standard UNIX implementations also
had this behavior.

  pe> Here is a patch that implements this suggestion; it assumes the
  pe> patches I sent in for GNU make 3.79.1 in July.  This patch also
  pe> fixes an EINTR-related bug in arscan.c that I discovered while
  pe> removing EINTR.

There is one problem with removing the EINTR handling: on some very old
systems we do not have any way to set restartable syscalls (as I
understand it).  The problem is these old systems don't have a
non-blocking wait(), so we have to install a SIGCHLD handler.

This is how the make code used to work back in the 3.75 and before
timeframe; we added the ability to use non-blocking wait and remove the
SIGCHLD handler on those systems that support it, but make still works
on older systems that don't.

Obviously, the support for this is somewhat half-baked, since there are
plenty of places where we _don't_ check for EINTR, but I think if you
don't use parallel builds it's relatively reliable.

I'm still not completely comfortable with abandoning those older
systems.

I suppose, though, we could always say that you just have to build an
older version of make on those older systems....

-- 
-------------------------------------------------------------------------------
 Paul D. Smith <address@hidden>          Find some GNU make tips at:
 http://www.gnu.org                      http://www.paulandlesley.org/gmake/
 "Please remain calm...I may be mad, but I am a professional." --Mad Scientist