bug#9768: Makefile broken after removing included *.am file

[Stefano Lattarini]

[me going through veeery old backlogs...]

References:
 <http://lists.gnu.org/archive/html/bug-automake/2009-11/msg00005.html>
 <http://lists.gnu.org/archive/html/automake-patches/2009-11/msg00008.html>

On Sunday 15 November 2009, Peter Johansson wrote:
> Hello,
>
Hi Peter and automakers, and sorry for the ludicrous delay of this reply.

> I included a file in my Makefile.am, but then I decided it was not 
> useful anymore so I removed the include statement and deleted the file. 
> That resulted in a broken Makefile, and running `make all' resulted in:
> 
> make: *** No rule to make target `aminclude.am', needed by 
> `Makefile.in'.  Stop.
> 
> This is very similar to the "deleted header file problem" for *.m4 files 
> that was fixed in version 1.11 by adding a stub rule. I suppose this 
> problem could be solved in the same fashion.
>
Thanks for the report and the suggestion; I've been bitten few times by
this bug myself, so I agree it's time to fix it.  See the attached
minimalistic patch, which I'll push to maint in a few days (as usual,
reviews welcome).  There is also a follow-up patch (attached as well)
that ensures the stub rules do not end up covering real errors.

With these two patches applied, the testsuite continues to pass both in
maint and master.

BTW, I see that the patches proposed by Ralf in the original threads (and
linked above) have never been applied; anybody (ping ping Ralf ;-) knows
why?

Regards,
  Stefano

From 0e8fe53d0ef3dac4901cecf5d576badb2cd6919b Mon Sep 17 00:00:00 2001
Message-Id: <address@hidden>
From: Stefano Lattarini <address@hidden>
Date: Sun, 16 Oct 2011 14:57:37 +0200
Subject: [PATCH 1/2] include: avoid "deleted .am file" problem

* lib/am/configure.am (%MAKEFILE-IN-DEPS%): New target without
dependencies, to avoid the "deleted .am file" problem.
* tests/remake-deleted-am.test: New test.
* tests/remake-deleted-am-2.test: New test.
* tests/remake-deleted-am-subdir.test: Likewise.
* tests/remake-renamed-am.test: Likewise.
* tests/Makefile.am (TESTS): Add them.
* NEWS: Update.

Report by Peter Johansson.

See also commit `Release-1-10-40-gd0ebf71', which fixed a similar
problem for .m4 files included by configure.ac.
---
 ChangeLog                           |   15 +++++++
 NEWS                                |    4 ++
 lib/am/configure.am                 |    3 +
 tests/Makefile.am                   |    4 ++
 tests/Makefile.in                   |    4 ++
 tests/remake-deleted-am-2.test      |   77 ++++++++++++++++++++++++++++++++++
 tests/remake-deleted-am-subdir.test |   79 +++++++++++++++++++++++++++++++++++
 tests/remake-deleted-am.test        |   69 ++++++++++++++++++++++++++++++
 tests/remake-renamed-am.test        |   60 ++++++++++++++++++++++++++
 9 files changed, 315 insertions(+), 0 deletions(-)
 create mode 100755 tests/remake-deleted-am-2.test
 create mode 100755 tests/remake-deleted-am-subdir.test
 create mode 100755 tests/remake-deleted-am.test
 create mode 100755 tests/remake-renamed-am.test

diff --git a/ChangeLog b/ChangeLog
index 171139c..65784a4 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,18 @@
+2011-10-16  Stefano Lattarini  <address@hidden>
+
+       include: avoid "deleted .am file" problem
+       * lib/am/configure.am (%MAKEFILE-IN-DEPS%): New target without
+       dependencies, to avoid the "deleted .am file" problem.
+       * tests/remake-deleted-am.test: New test.
+       * tests/remake-deleted-am-2.test: New test.
+       * tests/remake-deleted-am-subdir.test: Likewise.
+       * tests/remake-renamed-am.test: Likewise.
+       * tests/Makefile.am (TESTS): Add them.
+       * NEWS: Update.
+       Report by Peter Johansson.
+       See also commit `Release-1-10-40-gd0ebf71', which fixed a similar
+       problem for .m4 files included by configure.ac.
+
 2011-10-07  Stefano Lattarini  <address@hidden>
 
        parallel-tests: warn on conditional TEST_EXTENSIONS definition
diff --git a/NEWS b/NEWS
index 7e30ed0..7246754 100644
--- a/NEWS
+++ b/NEWS
@@ -68,6 +68,10 @@ Bugs fixed in 1.11.0a:
 
 * Long standing bugs:
 
+  - The "deleted header file problem" for *.am files is avoided by stub
+    rules.  This allows `make' to trigger a rerun of `automake' also if
+    some previously needed `.am' file has been removed.
+
   - "make distcheck" now correctly complains also when "make uninstall"
     leaves one and only one file installed in $(prefix).
 
diff --git a/lib/am/configure.am b/lib/am/configure.am
index 594ec67..753e09b 100644
--- a/lib/am/configure.am
+++ b/lib/am/configure.am
@@ -83,6 +83,9 @@ endif %?TOPDIR_P%
            cd $(top_builddir) && $(SHELL) ./config.status %CONFIG-MAKEFILE% 
$(am__depfiles_maybe);; \
        esac;
 
+## Avoid the "deleted header file" problem for the dependencies.
+%MAKEFILE-IN-DEPS%:
+
 DIST_COMMON += %MAKEFILE-AM%
 
 
diff --git a/tests/Makefile.am b/tests/Makefile.am
index 884ea65..af13b70 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -716,6 +716,10 @@ remake-subdir-gnu.test \
 remake-subdir.test \
 remake-subdir2.test \
 remake-subdir-long-time.test \
+remake-deleted-am.test \
+remake-deleted-am-2.test \
+remake-deleted-am-subdir.test \
+remake-renamed-am.test \
 pr8365-remake-timing.test \
 regex.test \
 req.test \
diff --git a/tests/Makefile.in b/tests/Makefile.in
index f1bfcdd..e085762 100644
--- a/tests/Makefile.in
+++ b/tests/Makefile.in
@@ -1000,6 +1000,10 @@ remake-subdir-gnu.test \
 remake-subdir.test \
 remake-subdir2.test \
 remake-subdir-long-time.test \
+remake-deleted-am.test \
+remake-deleted-am-2.test \
+remake-deleted-am-subdir.test \
+remake-renamed-am.test \
 pr8365-remake-timing.test \
 regex.test \
 req.test \
diff --git a/tests/remake-deleted-am-2.test b/tests/remake-deleted-am-2.test
new file mode 100755
index 0000000..654f390
--- /dev/null
+++ b/tests/remake-deleted-am-2.test
@@ -0,0 +1,77 @@
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# Check that inclusion of `.am' fragments by automake does not suffer
+# of the "deleted header problem".  This test checks deeper inclusion
+# stacks, and use VPATH builds.
+
+. ./defs || Exit 1
+
+set -e
+
+echo AC_OUTPUT >> configure.in
+
+$ACLOCAL
+$AUTOCONF
+
+cat > Makefile.am <<'END'
+include $(srcdir)/foo.am
+END
+
+cat > foo.am <<'END'
+# this is foo
+include sub/bar.am
+include baz.am
+END
+
+mkdir sub
+echo 'include $(top_srcdir)/sub/zardoz.am' > sub/bar.am
+echo '# this is zardoz' > sub/zardoz.am
+
+echo 'include fnord.am' > baz.am
+echo '# you are not seeing this' > fnord.am
+
+$AUTOMAKE
+# Sanity checks.
+$FGREP 'this is zardoz' Makefile.in
+$FGREP 'you are not seeing this' Makefile.in
+
+mkdir build
+cd build
+
+srcdir=..
+
+$srcdir/configure
+$MAKE
+
+$sleep
+echo '# this is baz' > $srcdir/baz.am
+rm -f $srcdir/fnord.am
+$MAKE
+# Sanity checks.
+$FGREP 'you are not seeing this' $srcdir/Makefile.in Makefile && Exit 1
+$FGREP 'this is baz' $srcdir/Makefile.in
+$FGREP 'this is baz' Makefile
+
+$sleep
+rm -rf $srcdir/sub $srcdir/foo.am $srcdir/baz.am 
+echo '# no more inclusions' > $srcdir/Makefile.am
+$MAKE
+# Sanity checks.
+$EGREP 'this is (foo|bar|baz)' Makefile $srcdir/Makefile.in && Exit 1
+$FGREP 'no more inclusions' Makefile
+
+:
diff --git a/tests/remake-deleted-am-subdir.test 
b/tests/remake-deleted-am-subdir.test
new file mode 100755
index 0000000..9ed1078
--- /dev/null
+++ b/tests/remake-deleted-am-subdir.test
@@ -0,0 +1,79 @@
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# Check that inclusion of `.am' fragments by automake does not suffer
+# of the "deleted header problem".  This test does the check when the
+# SUBDIRS variable is involved.
+
+. ./defs || Exit 1
+
+set -e
+
+cat >> configure.in <<'END'
+AC_CONFIG_FILES([sub1/Makefile sub2/Makefile])
+AC_OUTPUT
+END
+
+$ACLOCAL
+$AUTOCONF
+
+cat > Makefile.am <<'END'
+include foo.am
+SUBDIRS = sub1 sub2
+END
+
+echo '# this is foo.am' > foo.am
+
+mkdir sub1 sub2
+
+echo 'include $(srcdir)/bar.am' > sub1/Makefile.am
+echo '# this is bar.am' > sub1/bar.am
+
+echo 'include $(top_srcdir)/foo.am' > sub2/Makefile.am
+
+$AUTOMAKE
+# Sanity checks.
+$FGREP 'this is foo.am' Makefile.in
+$FGREP 'this is bar.am' sub1/Makefile.in
+$FGREP 'this is foo.am' sub2/Makefile.in
+
+./configure
+$MAKE # Should be no-op.
+
+$sleep
+echo '# this is sub1/Makefile.am' > sub1/Makefile.am
+rm -f sub1/bar.am
+$MAKE all
+# Sanity checks.
+$FGREP 'this is bar' sub1/Makefile.in sub1/Makefile && Exit 1
+$FGREP 'this is sub1/Makefile.am' sub1/Makefile.in
+$FGREP 'this is sub1/Makefile.am' sub1/Makefile
+
+$sleep
+for d in . sub2; do
+  sed "s|.*include.*foo\.am.*|# this is $d/Makefile.am|" $d/Makefile.am > t
+  mv -f t $d/Makefile.am
+done
+rm -f foo.am
+$MAKE all
+# Sanity checks.
+$FGREP 'this is foo' sub*/Makefile* Makefile* && Exit 1
+for d in . sub1 sub2; do
+  $FGREP "this is $d/Makefile.am" $d/Makefile.in
+  $FGREP "this is $d/Makefile.am" $d/Makefile
+done
+
+:
diff --git a/tests/remake-deleted-am.test b/tests/remake-deleted-am.test
new file mode 100755
index 0000000..fa5a672
--- /dev/null
+++ b/tests/remake-deleted-am.test
@@ -0,0 +1,69 @@
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# Check that inclusion of `.am' fragments by automake does not suffer
+# of the "deleted header problem".  Basic checks are done here.
+
+. ./defs || Exit 1
+
+set -e
+
+echo AC_OUTPUT >> configure.in
+
+$ACLOCAL
+$AUTOCONF
+
+cat > Makefile.am <<'END'
+include foo.am
+include $(srcdir)/bar.am
+include $(top_srcdir)/baz.am
+END
+
+echo '# this is foo' > foo.am
+echo '# this is bar' > bar.am
+echo '# this is baz' > baz.am
+
+$AUTOMAKE
+# Sanity checks.
+$FGREP 'this is foo' Makefile.in
+$FGREP 'this is bar' Makefile.in
+$FGREP 'this is baz' Makefile.in
+
+./configure
+$MAKE # Should be no-op.
+
+$sleep
+sed '/^include foo\.am$/d' Makefile.am > t
+mv -f t Makefile.am
+rm -f foo.am
+$MAKE Makefile
+# Sanity checks.
+$FGREP 'this is foo' Makefile.in Makefile && Exit 1
+$FGREP 'this is bar' Makefile.in
+$FGREP 'this is bar' Makefile
+$FGREP 'this is baz' Makefile.in
+$FGREP 'this is baz' Makefile
+
+$sleep
+echo '# empty empty' > Makefile.am
+rm -f bar.am baz.am
+$MAKE Makefile
+# Sanity checks.
+$FGREP 'empty empty' Makefile.in
+$FGREP 'empty empty' Makefile
+grep 'this is ba[rz]' Makefile Makefile.in && Exit 1
+
+:
diff --git a/tests/remake-renamed-am.test b/tests/remake-renamed-am.test
new file mode 100755
index 0000000..b3a7a41
--- /dev/null
+++ b/tests/remake-renamed-am.test
@@ -0,0 +1,60 @@
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# Check that inclusion of `.am' fragments by automake does not suffer
+# of the "deleted header problem".  This test checks that we can rename
+# an included .am file without confusing the remake rules.
+
+. ./defs || Exit 1
+
+set -e
+
+echo AC_OUTPUT >> configure.in
+
+cat > Makefile.am <<'END'
+include foo.am
+END
+
+cat > foo.am <<'END'
+all-local:
+       echo ok > has-run-1
+END
+
+$ACLOCAL
+$AUTOMAKE
+$AUTOCONF
+
+./configure
+$MAKE
+test -f has-run-1
+
+$sleep
+rm -f has-run-1
+mv -f foo.am bar.am
+echo include bar.am > Makefile.am
+$MAKE Makefile
+$FGREP 'foo.am' Makefile Makefile.in && Exit 1
+
+$sleep
+rm -f has-run-1 bar.am
+echo 'all-local:; echo ok > has-run-2' > zardoz.am
+echo 'include $(srcdir)/zardoz.am' > Makefile.am
+using_gmake || $MAKE Makefile
+$MAKE
+test ! -f has-run-1
+test -f has-run-2
+
+:
-- 
1.7.2.3

From 120eab98add774cef856c211b436dcb3719a791e Mon Sep 17 00:00:00 2001
Message-Id: <address@hidden>
In-Reply-To: <address@hidden>
References: <address@hidden>
From: Stefano Lattarini <address@hidden>
Date: Sun, 16 Oct 2011 17:07:19 +0200
Subject: [PATCH 2/2] coverage: required but missing '.am' and '.m4' files are 
diagnosed

The stub rules emitted to work around the "deleted header problem"
for `.m4' files (included by autoconf in e.g., configure.ac) and
for `.am' files (included by automake in e.g., Makefile.am) should
not prevent the remake rules from correctly erroring out when a
still-required file is missing.

* tests/deleted-am.test: New test.
* tests/deleted-m4.test: Likewise.
* tests/Makefile.am (TESTS): Add them.
---
 ChangeLog             |   12 +++++++++
 tests/Makefile.am     |    2 +
 tests/deleted-am.test |   58 +++++++++++++++++++++++++++++++++++++++++++
 tests/deleted-m4.test |   66 +++++++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 138 insertions(+), 0 deletions(-)
 create mode 100755 tests/deleted-am.test
 create mode 100755 tests/deleted-m4.test

diff --git a/ChangeLog b/ChangeLog
index 65784a4..e5cf3cd 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,17 @@
 2011-10-16  Stefano Lattarini  <address@hidden>
 
+       coverage: required but missing '.am' and '.m4' files are diagnosed
+       The stub rules emitted to work around the "deleted header problem"
+       for `.m4' files (included by autoconf in e.g., configure.ac) and
+       for `.am' files (included by automake in e.g., Makefile.am) should
+       not prevent the remake rules from correctly erroring out when a
+       still-required file is missing.
+       * tests/deleted-am.test: New test.
+       * tests/deleted-m4.test: Likewise.
+       * tests/Makefile.am (TESTS): Add them.
+
+2011-10-16  Stefano Lattarini  <address@hidden>
+
        include: avoid "deleted .am file" problem
        * lib/am/configure.am (%MAKEFILE-IN-DEPS%): New target without
        dependencies, to avoid the "deleted .am file" problem.
diff --git a/tests/Makefile.am b/tests/Makefile.am
index af13b70..3487d95 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -295,6 +295,8 @@ dejagnu4.test \
 dejagnu5.test \
 dejagnu6.test \
 dejagnu7.test \
+deleted-am.test \
+deleted-m4.test \
 depacl2.test \
 depcomp.test \
 depcomp2.test \
diff --git a/tests/deleted-am.test b/tests/deleted-am.test
new file mode 100755
index 0000000..29d104c
--- /dev/null
+++ b/tests/deleted-am.test
@@ -0,0 +1,58 @@
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# The stub rules emitted to work around the "deleted header problem"
+# for `.am' files shouldn't prevent the remake rules from correctly
+# erroring out when a still-required file is missing.
+
+. ./defs || Exit 1
+
+set -e
+
+echo AC_OUTPUT >> configure.in
+
+echo 'include $(top_srcdir)/foobar.am' > Makefile.am
+echo 'include zardoz.am' > foobar.am
+: > zardoz.am
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE
+
+./configure
+$MAKE
+
+rm -f zardoz.am
+$MAKE >output 2>&1 && { cat output; Exit 1; }
+cat output
+# This error will come from automake, not make, so we can be stricter
+# in our grepping of it.
+grep 'cannot open.*zardoz\.am' output
+grep 'foobar\.am' output && Exit 1 # No spurious error, please.
+
+# Try  with one less indirection.
+: > foobar.am
+$AUTOMAKE Makefile
+./config.status Makefile
+$MAKE # Sanity check.
+rm -f foobar.am
+$MAKE >output 2>&1 && { cat output; Exit 1; }
+cat output
+# This error will come from automake, not make, so we can be stricter
+# in our grepping of it.
+grep 'cannot open.*foobar\.am' output
+
+:
diff --git a/tests/deleted-m4.test b/tests/deleted-m4.test
new file mode 100755
index 0000000..3114bcd
--- /dev/null
+++ b/tests/deleted-m4.test
@@ -0,0 +1,66 @@
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# The stub rules emitted to work around the "deleted header problem"
+# for `.m4' files shouldn't prevent the remake rules from correctly
+# erroring out when a still-required file is missing.
+
+. ./defs || Exit 1
+
+set -e
+
+cat >> configure.in <<'END'
+m4_include([foobar.m4])
+AC_OUTPUT
+END
+
+: > Makefile.am
+
+echo 'm4_include([zardoz.m4])' > foobar.m4
+: > zardoz.m4
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE
+
+./configure
+$MAKE
+
+rm -f zardoz.m4
+$MAKE >output 2>&1 && { cat output; Exit 1; }
+cat output
+# This error will come from aclocal, not make, so we can be stricter
+# in our grepping of it.
+grep ' foobar\.m4:1:.*zardoz\.m4.*does not exist' output
+# No spurious errors, please.
+$FGREP -v ' foobar.m4:1:' output | $FGREP 'foobar.m4' && Exit 1
+
+# Try with one less indirection.
+: > foobar.m4
+$ACLOCAL --force
+$AUTOCONF
+./configure
+$MAKE # Sanity check.
+rm -f foobar.m4
+$MAKE >output 2>&1 && { cat output; Exit 1; }
+cat output
+# This error will come from aclocal, not make, so we can be stricter
+# in our grepping of it.
+grep 'foobar\.m4.*does not exist' output
+# No spurious errors, please (ok, this is really paranoid).
+$FGREP 'zardoz.m4' output && Exit 1
+
+:
-- 
1.7.2.3