Re: gettext-0.14.2 fails on parallel build

[Bruno Haible]

Alexandre Duret-Lutz wrote:
> > What happened was: In the rules
> >
> > elc-stamp: $(LISP)
> >         @rm -f elc-temp && touch elc-temp
> >         ...
> >         @mv -f elc-temp $@
> >
> > 3 independent 'make' processes started working on this rule.
> 
> It seems you are trying to allow this, but that is precisely the bug I
> wanted to fix.  The other points are consequences.  Only one instance
> of any rule in a Makefile should run at the same time.
> ...
> all instance are likely to use the same ressources
> (e.g. compiling the same files... eww).

Your modification does not fix this, because
  - It didn't change the dependencies of this rule, or the rules which
    depend on this one. Therefore this rule can still be started by two
    parallel processes.
  - The first line, "rm -f elc-temp && touch elc-temp", does nothing to
    prevent parallel execution.
  - Next, the test "if test "$(EMACS)" != no && test ! -f $@; "
    does not prevent parallel execution either, because it tests whether
    another process has already _finished_ its job, not whether it has
    already _begun_ its job. This scenario is still possible:

        Process 1 creates elc-temp.
        Process 2 re-creates elc-temp.
        Then process 1 and process 2 perform the "..." task.
        Process 1 moves elc-temp to elc-stemp.
        Process 2 attempts to do so as well, but elc-temp was already gone.

> > Adding "test "$(EMACS)" != no" doesn't change the problem.
> 
> It gets rid of these 3 independent 'make' processes.  (Those
> are not started when emacs exists, because when emacs exists
> the *.elc files have already been built and the `test ! -f $@'
> above fails.)

The first time this rule is executed, after the programmer has freshly
written several *.elc files, the problem is still there.

> Here is another angle to the problem that might help: this whole issue
> would not exist if the `$(am__ELCFILES): elc-stamp' rule did not have
> any command.

Let's consider the heart of the problem: In a parallel "make", we want to
avoid simultaneous execution of the same commands of a single target. For
this, we need
  1) a way to designate one of the several processes as the "first" one
     and the other ones as "followers",
  2) a way to let the "followers" wait until the "first" one has done his
     job.
For 1) we need an atomic operation, such as "mv" or "ln".
For 2) a loop that executes "sleep 1" is enough.
So here comes [my 3rd attempt at] a fix:

elc-stamp: $(LISP)
    pid=$$$$; \
    trap 'rm -f elc-$$pid elc-temp elc-stamp' 1 2 3 15; \
    touch elc-$$pid; \
    if ln elc-$$pid elc-temp 2>/dev/null; then \
      # Comment: This code is being executed by the first process.
      rm -f elc-$$pid; \
      if test "$(EMACS)" != no; then \
        [compile the *.lisp files] \
      else : ; fi; \
      touch $@; \
      rm -f elc-temp; \
    else \
      # Comment: This code is being executed by the follower processes.
      rm -f elc-$$pid; \
      # Comment: Wait until the first process is done.
      while test -f elc-temp; do sleep 1; done; \
      # Comment: Succeed if and only if the first process succeeded.
      test -f $@; exit $$?; \
    fi

The only nit on this fix is that it doesn't work on filesystems that don't
support hard links (like the BeOS filesystem).

Bruno