bug#8365: 3 of 657 tests failed

[Stefano Lattarini]

Hi Ralf.  I want to defend and reiterate my explanation, and this
time I'll be able to back it up with some real data (thanks to Sam
for providing it so quickly!) and a new testcase.

On Wednesday 30 March 2011, Ralf Wildenhues wrote:
> * Ralf Wildenhues wrote on Wed, Mar 30, 2011 at 06:59:07PM CEST:
> > * Stefano Lattarini wrote on Wed, Mar 30, 2011 at 06:47:20PM CEST:
> > > --- a/tests/subdir5.test
> > > +++ b/tests/subdir5.test
> > 
> > > @@ -67,6 +67,8 @@ $MAKE
> > >  # does it in the other way: it updates confiles.m4 (which is m4_included
> > >  # by configure.in there) after Makefile.am.
> > >  
> > > +# Modified configure dependencies must be newer than config.status.
> > > +$sleep
> > 
> > This hunk makes perfect sense to me, and is fine for maint,
> 
> Never mind; this hunk has the same problem as the other two.
> Cf. also the _long_ comment above it explaining why it should
> be sufficient.
>
Yeah, sorry for missing that.  By doing so I've probably lengthened
and complicated this discussion :-(

> This comment would need fixing if it were wrong.
>
It's not wrong per se, just misleading.

It reads:

 # We shouldn't need to $sleep here: configure ensures that files
 # generated by it are newer than configure.  Thus, even if
 # Makefile.in is newer than configure but the updated Makefile.am
 # below has the same timestamp as Makefile.in, the latter should
 # be rebuilt due to its dependency on configure.in.

Well, the point here is that, while Makefile.in *is* indeed rebuilt,
Makefile is *not* -- precisely because it depends only on config.status
and Makefile.in, and in our scenario, they've have both the same
timestamp of Makefile.

The attached testcase exposes this bug consistently and portably
(through some hops and hacks).  It might be particularly instructive
to run it with MAKE='gmake -r --debug=a', to see what rules and
dependencies are considered, and to check that the observed behaviour
is indeed consistent with the explanation/considerations given below.

-*-*-

Let's now walk through the on-field failure again (with the backup
of the content of Sam's test directories this time).  Sorry if I'm
being overly explicit and verbose below, but I want to be sure we
pinpoint the reason of this failure correctly.

> FAIL: subdir5.test (exit: 1)
> ============================
>
> /home/sds/src/automake-1.11.1/tests:/home/sds/src/top/bin:/usr/lib64
> /qt-3.3/bin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/usr/NX/b
> in:/home/sds/bin:/sbin:/usr/sbin:/usr/NX/bin:/usr/NX/bin:/home/sds/b
> in:/sbin:/usr/sbin:/usr/NX/bin:/home/sds/bin:/sbin:/usr/sbin
>
> subdir5: running make --version -v | grep GNU
> GNU Make 3.81
> subdir5: running gcc --version
> gcc (GCC) 4.1.2 20080704 (Red Hat 4.1.2-48)
> Copyright (C) 2006 Free Software Foundation, Inc.
> This is free software; see the source for copying conditions.  There
> is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
> PARTICULAR PURPOSE.
>
> === Running test ./subdir5.test
> ++ pwd
> /home/sds/src/automake-1.11.1/tests/subdir5.dir
> + set -e
> + cat
> + cat
> + cat
> + aclocal-1.11 -Werror
> + autoconf
> + automake-1.11 --foreign -Werror -Wall --include-deps --copy --add-missing
> configure.in:3: installing `./compile'
> + ./configure
> checking for a BSD-compatible install... /usr/bin/install -c
> checking whether build environment is sane... yes
> checking for a thread-safe mkdir -p... /bin/mkdir -p
> checking for gawk... gawk
> checking whether make sets $(MAKE)... yes
> checking for style of include used by make... GNU
> checking for gcc... gcc
> checking whether the C compiler works... yes
> checking for C compiler default output file name... a.out
> checking for suffix of executables...
> checking whether we are cross compiling... no
> checking for suffix of object files... o
> checking whether we are using the GNU C compiler... yes
> checking whether gcc accepts -g... yes
> checking for gcc option to accept ISO C89... none needed
> checking dependency style of gcc... gcc3
> checking whether gcc and cc understand -c and -o together... yes
> configure: creating ./config.status
> config.status: creating Makefile
> config.status: executing depfiles commands
> + make
> gcc -DPACKAGE_NAME=\"maude\" -DPACKAGE_TARNAME=\"maude\"  \
>     -DPACKAGE_VERSION=\"1.0\" -DPACKAGE_STRING=\"maude\ 1.0\" \
>     -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"maude\" \
>     -DVERSION=\"1.0\" -I. -g -O2 -MT a.o -MD -MP -MF .deps/a.Tpo -c \
>     -o a.o a.c mv -f .deps/a.Tpo .deps/a.Po
> gcc  -g -O2   -o wish a.o
> + cat
> + :
> + mkdir maude
> + cat
> + :
> + echo 'SUBDIRS = maude'
> + make
> CDPATH="${ZSH_VERSION+.}:" && cd . && /bin/sh 
> /home/sds/src/automake-1.11.1/tests/subdir5.dir/missing --run aclocal-1.11
>  cd . && /bin/sh /home/sds/src/automake-1.11.1/tests/subdir5.dir/missing 
> --run automake-1.11 --foreign
> CDPATH="${ZSH_VERSION+.}:" && cd . && /bin/sh 
> /home/sds/src/automake-1.11.1/tests/subdir5.dir/missing --run autoconf
>
Correct so far: aclocal.m4 dependencies are modified, so the
Automake-generated Makefile re-runs aclocal, autoconf and automake ...

> make: Nothing to be done for `all'.
>
... but then it doesn't re-run configure!  So that the newly added makefile
`maude/Makefile' is not built, and consequently ...

> + test -f maude/Makefile
> + exit_status=1
>
... this test fails.

Now, why is configure not re-run?  Here is my  explanation, step by step.
In what follows, I'll use the abbraviation `TS(f)' to indicate the
timestamp (i.e. last modification time) of the file `f'.

 01. The autotools are run to create `aclocal.m4', `configure' and
     `Makefile.in'; on a fast machine, automake and autoconf take very
     little time to run on small input files, so that it is very likely
     that:
       [E1.1] TS(configure) = TS(Makefile.in)
     and we'll assume this is indeed the case.
     Also, aclocal is run before both automake and autoconf, so that it
     will surely be:
       [E1.2] TS(aclocal.m4) <= TS(configure)
       [E1.3] TS(aclocal.m4) <= TS(Makefile.in)

 02. `configure' is run for the first time; it creates the `config.status'
     script and the `Makefile', which are both assured to be created newer
     than configure (see code in sanity.m4:AM_SANITY_CHECK):
       [E2.1] TS(config.status) > TS(configure)
       [E2.2] TS(Makefile) > TS(configure)

 03. Since the creations of `config.status' and `Makefile' take place very
     near, it's quite likely they'll have the same timestamp; let's assume
     this is indeed the case:
       [E3] TS(config.status) = TS(Makefile)

 04. make is run for the first time; all is up-to-date at this point, so that
     nothing gets rebuilt -- basically a no-op;

 05. the test script creates `maude/Makefile.am, modifies `configure.in' by
     adding `maude/Makefile' to AC_CONFIG_FILES, and modifies `Makefile.am'
     by defining `SUBDIRS' to `maude', *without sleeping* before doing these
     modifications.  This lack of a `sleep' is very important; it means that,
     on a fast-enoough machine, `Makefile.am' and `configure.in' can still
     have the same timestamp of `config.status' and `Makefile'; let's assume
     this is the case, so we'll get:
       [E5] TS(config.status) = TS(Makefile) = TS(configure.in) =
            = TS(Makefile.am)

 06. make is called.  It has to check if `Makefile' is out-of-date, and
     in that case, to rebuild it.

 07. `configure' depends on `configure.in', by [E5] and [E2.1] we have:
       [E7.1] TS(configure) < TS(configure.in)
     so that `configure' is out-of-date; also, `config.status' depends on
     `configure', and `Makefile' depends on `config.status', so make will
     set out to rebuild `configure'.

 08. `configure' depends on `aclocal.m4', `aclocal.m4' depends on
     `configure.in', and by [E7.1] and [E1.2] we have:
       [E8.1] S(aclocal.m4) < TS(configure.in)
     so, for the same reasons explained in the previous point (7), make
     will set out to rebuild `aclocal.m4'.

 09. `Makefile.in' depends on `Makefile.am' and `configure.in', and by
      [E5], [E2.1] [E1.1] we have:
        [E9.1] TS(Makefile.in) < TS(Makefile.am) 
        [E9.2] TS(Makefile.in) < TS(configure.in) 
     so that `Makefile.in' is out-of-date; also, `Makefile' depends on it,
     so in the end make will set out to rebuild `Makefile.in'.

 10. Given (7), (8) and (9), make issues calls to aclocal, autoconf and
     automake in order to bring up-to-date the various files that `Makefile'
     depends on. After the autotools have run, we have:
      [E10] for each x in { Makefile.in, configure, aclocal.m4 }
            for each y in { config.status, Makefile }
            it is TS(x) >= TS(y)

 11. Now, since no explicit call to `sleep' has been issued in any of the
     preceding steps, it's *possible* (assuming those steps have been
     executed fast enough, as would mayhap be possible on a fast, lighlty
     loaded system) that all the files updated until now (the first set
     in [E10]) still have the same timestamp of `config.status'.  Let's
     assume this is indeed the case; so we have:
       [E11] TS(Makefile.in) = TS(configure) = TS(aclocal.m4) =
             = TS(config.status) = TS(Makefile)

 12. In virtue of [E11], make now sees that `config.status' is *not*
     older than `configure'; so `config.status --recheck' (which would
     be responsible to re-run configure, and regenerate an updated
    `config.status') won't be called.

 12. Since `Makefile' depends *only* on `Makefile.in' and `config.status',
     and these are up-to-date as per [E11] and the observation in previous
     point (12), `Makefile' won't be regenerated either (it would have been
     if `config.status --recheck' had been run).

 13. Failure ensues. D'oh!

-*-*-

Now let's see if the above analysis is backed up by facts.  If the
analysis is correct, then:
  F1) [E11] must hold, i.e. Makefile.in, configure, aclocal.m4,
      config.status and Makefile must all have the same timestamp;
  F2) `Makefile.in' has been updated, so it will contain the
      definition of SUBDIRS to `maude';
  F3) `Makefile' has *not* been updated, so it won't contain any
      SUBDIRS deifnition;
  F4) `configure' has been updated, so it will list `maude/Makefile'
      among $ac_config_files
  F5) `config.status' has *not* been updated, so it won't contain
      any mention to `maude/Makefile'.

And all this fact hold for the test directory Sam has kindly sent me:

  $ wget http://sds.podval.org/data/automake-1.11.1-dirs.tgz
  $ tar xzvf automake-1.11.1-dirs.tgz
  $ cd automake-1.11.1/tests/subdir5.dir
  ## all the relevant files must have the same timestamp, i.e., [E11] must hold
  $ stat --format='%y -- %n'  configure.in Makefile.in configure config.status 
Makefile
  2011-03-28 16:14:15.000000000 +0200 -- configure.in
  2011-03-28 16:14:15.000000000 +0200 -- Makefile.in
  2011-03-28 16:14:15.000000000 +0200 -- configure
  2011-03-28 16:14:15.000000000 +0200 -- config.status
  2011-03-28 16:14:15.000000000 +0200 -- Makefile
  ## Makefile.in and configure must reference the maude subdir
  $ grep '^SUBDIRS *=' Makefile.in
  SUBDIRS = maude
  $ grep 'maude/Makefile' configure
  ac_config_files="$ac_config_files Makefile maude/Makefile"
      "maude/Makefile") CONFIG_FILES="$CONFIG_FILES maude/Makefile" ;;
  ## Makefile and config.status must not reference the maude subdirs
  $ grep SUBDIRS Makefile || echo No match
  No match
  $ grep maude/Makefile config.status || echo No match
  No match
  
-*-*-

At this point I'm not anymore sure this is just a testsuite-related issue
-- it seems like a genuine bug in Automake-generated remake rules.  WDYT?

Regards,
  Stefano

remake-bug8365.test
Description: application/shellscript