[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Automake-ng] [PATCH 03/10] [ng] parallel-tests: simplify automake-time
|
From: |
Stefano Lattarini |
|
Subject: |
[Automake-ng] [PATCH 03/10] [ng] parallel-tests: simplify automake-time preprocessing |
|
Date: |
Mon, 23 Apr 2012 11:57:21 +0200 |
Our new pattern rules, introduced in recent commit 'v1.11b-124-gaeaba3f',
are smart enough to allow us to ditch most of our cumbersome automake-time
analysis and rewriting of the $(TESTS) variable when the parallel-tests
driver is used. This also makes other related simplifications possible.
This change causes a little regression in that we won't be able anymore to
diagnose at automake time test cases that start with the '$(srcdir)' or
'$(top_srcdir)' component, but that's acceptable, since a later commit will
make the former case actually work, and the second case isn't important
enough to worry to much about.
This change also causes a small behavioural change in automake, in that it
will also need the 'LOG_DRIVER' variable to be defined in each Makefile.am
defining the 'TESTS' variable; and if it's not defined, automake will
define it itself, and require the presence of the 'test-driver' auxiliary
script in the process (or bing it in if the "--add-missing" option is in
use). See the changes to 'NG-NEWS' for more details, and for workarounds.
* automake.in (handle_tests): Don't walk nor rewrite $(TESTS) when the
'parallel-tests' option is active. Other related minor simplifications.
* t/exeext4.sh: Define '$am_parallel_tests' to "no", so that the test
won't be run with the 'parallel-tests' option enabled (as the semantics
in that case have been changed in a way that would make this test fail).
* t/check7.sh: When '$am_parallel_tests' is "yes", skip checks that
don't work anymore with the 'parallel-tests' option is enabled.
* t/test-driver-custom-no-extra-driver.sh: Adjust to avoid spurious
failures to to the "LOG_DRIVER must be defined somehow" semantic change.
* t/tap-common-setup.sh: Likewise.
* t/tap-recheck.sh: Likewise.
* t/tap-basic.sh: Likewise.
* t/tap-diagnostic-custom.sh: Likewise.
* t/tap-more.sh: Likewise.
* t/tap-recheck.sh: Likewise.
* t/tap-signal.tap: Likewise.
* t/test-driver-custom-multitest.sh: Likewise.
* t/test-driver-custom-multitest-recheck.sh: Likewise.
* t/test-driver-custom-multitest-recheck2.sh: Likewise.
* t/test-driver-custom-xfail-tests.sh: Likewise.
* t/test-driver-fail.sh: Likewise.
* t/test-driver-strip-vpath.sh: Likewise.
* t/test-metadata-global-log.sh: Likewise.
* t/test-metadata-global-results.sh: Likewise, and enhanced a little since
we are at it.
* t/test-metadata-results.sh: Likewise.
* t/tap-bad-prog.tap: Likewise.
* t/parallel-tests8.sh: Move the now-failing checks verifying that automake
diagnoses test cases that start with the '$(srcdir)' or '$(top_srcdir)'...
* t/parallel-tests-srcdir-in-test-name.sh: ... this new xfailing test.
* t/spy-pattern-rules.sh: New "spy" test, verify our new assumptions on the
precedence of pattern rules truly hold.
* t/parallel-tests-longest-stem.sh: New test.
* t/list-of-tests.mk: Update.
* doc/automake.texi: Document that $(TESTS) is rewritten for $(EXEEXT)
appending only when the 'parallel-tests' option is *not* in use.
Add a "FIXME" comment about the fact that automake does not diagnose test
cases that start with the '$(srcdir)' or '$(top_srcdir)' anymore.
* NG-NEWS: Update.
Signed-off-by: Stefano Lattarini <address@hidden>
---
NG-NEWS | 22 ++++++
automake.in | 108 ++++++++++------------------
doc/automake.texi | 6 +-
t/check7.sh | 9 ++-
t/exeext4.sh | 1 +
t/list-of-tests.mk | 4 ++
t/parallel-tests-longest-stem.sh | 66 +++++++++++++++++
t/parallel-tests-srcdir-in-test-name.sh | 41 +++++++++++
t/parallel-tests8.sh | 18 +----
t/spy-pattern-rules.sh | 54 ++++++++++++++
t/tap-bad-prog.tap | 13 ++--
t/tap-basic.sh | 2 +
t/tap-common-setup.sh | 8 ++-
t/tap-diagnostic-custom.sh | 1 +
t/tap-more.sh | 3 +-
t/tap-recheck.sh | 3 +-
t/tap-signal.tap | 2 +-
t/test-driver-custom-multitest-recheck.sh | 2 +-
t/test-driver-custom-multitest-recheck2.sh | 2 +-
t/test-driver-custom-multitest.sh | 2 +
t/test-driver-custom-no-extra-driver.sh | 5 +-
t/test-driver-custom-xfail-tests.sh | 2 +-
t/test-driver-fail.sh | 8 ++-
t/test-driver-strip-vpath.sh | 11 +--
t/test-metadata-global-log.sh | 1 +
t/test-metadata-global-result.sh | 45 +++++++++++-
t/test-metadata-results.sh | 25 +++----
27 files changed, 334 insertions(+), 130 deletions(-)
create mode 100755 t/parallel-tests-longest-stem.sh
create mode 100755 t/parallel-tests-srcdir-in-test-name.sh
create mode 100755 t/spy-pattern-rules.sh
diff --git a/NG-NEWS b/NG-NEWS
index 152fecc..62c666a 100644
--- a/NG-NEWS
+++ b/NG-NEWS
@@ -71,6 +71,28 @@ Parallel testsuite harness
variables anymore. These have always been meant for internal use only
anyway.
+* The $(TESTS) and $(XFAIL_TESTS) variables are not anymore rewritten for
+ $(EXEEXT) appending. The use of compiled programs in $(TESTS) still
+ works as before though, even on systems where $(EXEEXT) is non-empty,
+ thanks to a careful use of pattern rules.
+
+* The new implementation of the parallel-tests harness tries to defer
+ as much as possible processing on the $(TESTS) variable at make,
+ rather than doing it at automake time. This as some positive effects
+ (FIXME: describe and quantify them), but also entails a minor annoyance.
+ When automake runs, it cannot know anymore whether $(TESTS) contains
+ or not any entry with no recognized test suffix; as a consequence, it
+ always need the '$(LOG_DRIVER)' variable to be defined. If it's not,
+ automake will define it to a proper invocation of the 'test-driver'
+ auxiliary script, and require that script to be present in the
+ config-aux directory (or bring it in if "--add-missing" is in use).
+ If you know you package won't make use of suffix-less tests (thus, in
+ particular, no use of compiled programs in $(TESTS)), you can avoid
+ requiring the 'test-driver' script by adding this line to your
+ configure.ac:
+ AC_SUBST([LOG_DRIVER], ['unused but required by automake'])
+
+
Miscellaneous
=============
diff --git a/automake.in b/automake.in
index 64ec53d..6b9cb94 100644
--- a/automake.in
+++ b/automake.in
@@ -4781,27 +4781,12 @@ sub handle_tests
PARALLEL_TESTS => !! option
'parallel-tests',
CHECK_DEPS => $check_deps);
- # Tests that are known programs should have $(EXEEXT) appended.
- # For matching purposes, we need to adjust XFAIL_TESTS as well.
- append_exeext { exists $known_programs{$_[0]} } 'TESTS';
- append_exeext { exists $known_programs{$_[0]} } 'XFAIL_TESTS'
- if (var ('XFAIL_TESTS'));
-
if (my $parallel_tests = option 'parallel-tests')
{
define_variable ('TEST_SUITE_LOG', 'test-suite.log', INTERNAL);
my $suff = '.test';
- my $at_exeext = '';
- my $handle_exeext = exists $configure_vars{'EXEEXT'};
- if ($handle_exeext)
- {
- $at_exeext = subst ('EXEEXT');
- $suff = $at_exeext . ' ' . $suff;
- }
- if (! var 'TEST_EXTENSIONS')
- {
- define_variable ('TEST_EXTENSIONS', $suff, INTERNAL);
- }
+ define_variable ('TEST_EXTENSIONS', '.test', INTERNAL)
+ if (! var 'TEST_EXTENSIONS');
my $var = var 'TEST_EXTENSIONS';
# Currently, we are not able to deal with conditional contents
# in TEST_EXTENSIONS.
@@ -4818,68 +4803,47 @@ sub handle_tests
"invalid test extensions: @invalid_test_suffixes";
}
@test_suffixes = grep { is_valid_test_extension $_ } @test_suffixes;
- if ($handle_exeext)
- {
- unshift (@test_suffixes, $at_exeext)
- unless $test_suffixes[0] eq $at_exeext;
- }
- unshift (@test_suffixes, '');
-
- transform_variable_recursively
- ('TESTS', 'TEST_LOGS', 'am__testlogs', 1, INTERNAL,
- sub {
- my ($subvar, $val, $cond, $full_cond) = @_;
- my $obj = $val;
- return $obj
- if $val =~ /address@hidden@$/;
- $obj =~ s/\$\(EXEEXT\)$//o;
-
- if ($val =~ /(\$\((top_)?srcdir\))\//o)
- {
- msg ('error', $subvar->rdef ($cond)->location,
- "parallel-tests: using '$1' in TESTS is currently
broken: '$val'");
- }
-
- foreach my $test_suffix (@test_suffixes)
- {
- next
- if $test_suffix eq $at_exeext || $test_suffix eq '';
- return substr ($obj, 0, length ($obj) - length
($test_suffix)) . '.log'
- if substr ($obj, - length ($test_suffix)) eq $test_suffix;
- }
- handle_per_suffix_test ('');
- return "$obj.log";
- });
-
- my $nhelper=1;
- my $prev = 'TESTS';
- my $post = '';
- my $last_suffix = $test_suffixes[$#test_suffixes];
- my $cur = '';
+ # It is *imperative* that the empty 'suffix goes last.
+ # Otherwise, a declaration like "TESTS = all.test" would cause GNU
+ # make to mistakenly try to build the 'all.log' and 'all.trs' files
+ # from a non-existent 'all' program (because the Makefile contains
+ # an explicit 'all' target) rather than from the 'all.test' script,
+ # thus causing all sort of mishaps and confusion.
+ push @test_suffixes, '';
+
+ define_variable ('am__test_logs1',
+ '$(patsubst %,%.log,$(strip $(TESTS)))',
+ INTERNAL);
+ my $nhelper = 1;
foreach my $test_suffix (@test_suffixes)
{
- if ($test_suffix eq $last_suffix)
- {
- $cur = 'TEST_LOGS';
- }
- else
- {
- $cur = 'am__test_logs' . $nhelper;
- }
- define_variable ($cur,
- '$(' . $prev . ':' . $test_suffix . $post . '=.log)', INTERNAL);
- $post = '.log';
- $prev = $cur;
- $nhelper++;
- if ($test_suffix ne $at_exeext && $test_suffix ne '')
- {
- handle_per_suffix_test ($test_suffix);
- }
+ handle_per_suffix_test ($test_suffix);
+ my @cooked_test_suffixes = ($test_suffix);
+ push @cooked_test_suffixes, ($test_suffix . '$(EXEEXT)')
+ if exists $configure_vars{'EXEEXT'};
+ foreach my $cooked_test_suffix (@cooked_test_suffixes)
+ {
+ my $old_var = 'am__test_logs' . $nhelper++;
+ my $cur_var = 'am__test_logs' . $nhelper;
+ define_variable (
+ $cur_var,
+ "\$($old_var:$cooked_test_suffix.log=.log)",
+ INTERNAL);
+ }
}
+ define_variable ('TEST_LOGS', "\$(am__test_logs$nhelper)", INTERNAL);
$clean_files{'$(TEST_LOGS)'} = MOSTLY_CLEAN;
$clean_files{'$(TEST_LOGS:.log=.trs)'} = MOSTLY_CLEAN;
$clean_files{'$(TEST_SUITE_LOG)'} = MOSTLY_CLEAN;
}
+ else
+ {
+ # Tests that are known programs should have $(EXEEXT) appended.
+ # For matching purposes, we need to adjust XFAIL_TESTS as well.
+ append_exeext { exists $known_programs{$_[0]} } 'TESTS';
+ append_exeext { exists $known_programs{$_[0]} } 'XFAIL_TESTS'
+ if (var ('XFAIL_TESTS'));
+ }
}
}
diff --git a/doc/automake.texi b/doc/automake.texi
index 2661af5..d1e0765 100644
--- a/doc/automake.texi
+++ b/doc/automake.texi
@@ -6938,8 +6938,9 @@ bin_PROGRAMS = liver$(EXEEXT)
The targets Automake generates are likewise given the @samp{$(EXEEXT)}
extension.
-The variables @code{TESTS} and @code{XFAIL_TESTS} (@pxref{Simple Tests})
-are also rewritten if they contain filenames that have been declared as
+When the @option{parallel-tests} option is in use, the variables
address@hidden and @code{XFAIL_TESTS} (@pxref{Simple Tests}) are also
+rewritten if they contain filenames that have been declared as
programs in the same @file{Makefile}. (This is mostly useful when some
programs from @code{check_PROGRAMS} are listed in @code{TESTS}.)
@@ -9202,6 +9203,7 @@ the inference rules generated by @command{automake} can
apply. For
literal test names, @command{automake} can generate per-target rules
to avoid this limitation.
address@hidden FIXME: Automake does not warn about these anymore!
Please note that it is currently not possible to use @code{$(srcdir)/}
or @code{$(top_srcdir)/} in the @code{TESTS} variable. This technical
limitation is necessary to avoid generating test logs in the source tree.
diff --git a/t/check7.sh b/t/check7.sh
index 1003b55..962cbe0 100755
--- a/t/check7.sh
+++ b/t/check7.sh
@@ -59,9 +59,12 @@ $AUTOMAKE -a
./configure
$MAKE check
-$MAKE EXEEXT=.bin print-xfail-tests >stdout || { cat stdout; Exit 1; }
-cat stdout
-$FGREP 'BEG: a.bin b c.bin d.bin :END' stdout
+
+if test x"$am_parallel_tests" != x"yes"; then
+ $MAKE EXEEXT=.bin print-xfail-tests >stdout || { cat stdout; Exit 1; }
+ cat stdout
+ $FGREP 'BEG: a.bin b c.bin d.bin :END' stdout
+fi
$MAKE distcheck
diff --git a/t/exeext4.sh b/t/exeext4.sh
index 298879d..53c37df 100755
--- a/t/exeext4.sh
+++ b/t/exeext4.sh
@@ -17,6 +17,7 @@
# Make sure $(EXEEXT) is appended to programs and to tests that are
# programs, but not to @address@hidden
+am_parallel_tests=no
. ./defs || Exit 1
cat >> configure.ac << 'END'
diff --git a/t/list-of-tests.mk b/t/list-of-tests.mk
index 436e401..7d12d0f 100644
--- a/t/list-of-tests.mk
+++ b/t/list-of-tests.mk
@@ -40,6 +40,7 @@ t/instdir-cond2.sh \
t/interp3.sh \
t/java-nobase.sh \
t/objext-pr10128.sh \
+t/parallel-tests-srcdir-in-test-name.sh \
t/parallel-tests-many.sh \
t/pr8365-remake-timing.sh \
t/remake-am-pr10111.sh \
@@ -738,6 +739,8 @@ t/parallel-tests8.sh \
t/parallel-tests9.sh \
t/parallel-tests10.sh \
t/parallel-tests-no-repeat.sh \
+t/parallel-tests-longest-stem.sh \
+t/parallel-tests-srcdir-in-test-name.sh \
t/parallel-tests-recheck-depends-on-all.sh \
t/parallel-tests-exeext.sh \
t/parallel-tests-suffix.sh \
@@ -981,6 +984,7 @@ t/spell3.sh \
t/spelling.sh \
t/spy-autovars.sh \
t/spy-double-colon.sh \
+t/spy-pattern-rules.sh \
t/spy-phony.sh \
t/spy-rm.tap \
t/spy-vpath-rewrite.sh \
diff --git a/t/parallel-tests-longest-stem.sh b/t/parallel-tests-longest-stem.sh
new file mode 100755
index 0000000..242d78f
--- /dev/null
+++ b/t/parallel-tests-longest-stem.sh
@@ -0,0 +1,66 @@
+#! /bin/sh
+# Copyright (C) 2012 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# The parallel-tests driver must prefer tests with an extension to
+# extension-less tests. This is required to allow the user to have
+# a, say, 'all.test' test case even in the face of the 'all' target.
+
+am_parallel_tests=yes
+. ./defs || Exit 1
+
+echo AC_OUTPUT >> configure.ac
+
+cat > foo <<'END'
+#!/bin/sh
+echo "foo without suffix run" >&2
+exit 99
+END
+
+cat > foo.test <<'END'
+#!/bin/sh
+echo "$0 has been run"
+END
+chmod a+x foo.test
+
+cp foo.test all.test
+cp foo.test dist.test
+cp foo.test install.test
+cp foo.test bad-target.test
+
+
+cat > Makefile.am << 'END'
+bad-target:
+ @echo $@ has been run >&2; exit 1
+install-data-local:
+ @echo $@ has been run >&2; exit 1
+TESTS = foo.test all.test install.test dist.test bad-target.test
+EXTRA_DIST = oops-this-does-not-exist
+END
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+
+./configure --prefix="`pwd`/inst"
+
+$MAKE check
+ls -l # For debugging.
+test ! -d inst
+for t in foo all install dist bad-target; do
+ grep "$t\.test has been run" $t.log
+done
+
+:
diff --git a/t/parallel-tests-srcdir-in-test-name.sh
b/t/parallel-tests-srcdir-in-test-name.sh
new file mode 100755
index 0000000..01042a5
--- /dev/null
+++ b/t/parallel-tests-srcdir-in-test-name.sh
@@ -0,0 +1,41 @@
+#! /bin/sh
+# Copyright (C) 2009-2012 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Check parallel-tests features:
+# - listing $(srcdir)/ or $(top_srcdir)/ in TESTS doesn't work ATM,
+# and is thus diagnosed.
+
+# TODO: this test should also ensure that the 'make' implementation
+# properly adheres to rules in all cases. See the Autoconf
+# manual for the ugliness in this area, when VPATH comes into
+# play. :-/
+
+am_parallel_tests=yes
+. ./defs || Exit 1
+
+echo AC_OUTPUT >> configure.ac
+
+cat > Makefile.am << 'END'
+TESTS = $(srcdir)/bar.test $(top_srcdir)/baz.test
+END
+
+$ACLOCAL
+$AUTOCONF
+AUTOMAKE_fails -a
+grep '$(srcdir).*TESTS.*bar\.test' stderr
+grep '$(top_srcdir).*TESTS.*baz\.test' stderr
+
+:
diff --git a/t/parallel-tests8.sh b/t/parallel-tests8.sh
index 69efc51..a36cea7 100755
--- a/t/parallel-tests8.sh
+++ b/t/parallel-tests8.sh
@@ -14,15 +14,7 @@
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
-# Check parallel-tests features:
-# - generated distributed tests.
-# - listing $(srcdir)/ or $(top_srcdir)/ in TESTS doesn't work ATM,
-# and is thus diagnosed.
-
-# TODO: this test should also ensure that the 'make' implementation
-# properly adheres to rules in all cases. See the Autoconf
-# manual for the ugliness in this area, when VPATH comes into
-# play. :-/
+# Check parallel-tests features: generated distributed tests.
am_parallel_tests=yes
. ./defs || Exit 1
@@ -33,8 +25,6 @@ END
cat > Makefile.am << 'END'
TESTS = foo.test
-## the next line will cause automake to error out:
-TESTS += $(srcdir)/bar.test $(top_srcdir)/baz.test
.in.test:
cp $< $@ && chmod +x $@
check_SCRIPTS = $(TESTS)
@@ -50,12 +40,6 @@ END
$ACLOCAL
$AUTOCONF
-AUTOMAKE_fails -a
-grep '$(srcdir).*TESTS.*bar\.test' stderr
-grep '$(top_srcdir).*TESTS.*baz\.test' stderr
-
-sed '/^TESTS +=.*srcdir/d' < Makefile.am > t
-mv -f t Makefile.am
$AUTOMAKE -a
./configure
diff --git a/t/spy-pattern-rules.sh b/t/spy-pattern-rules.sh
new file mode 100755
index 0000000..b2bb25d
--- /dev/null
+++ b/t/spy-pattern-rules.sh
@@ -0,0 +1,54 @@
+#! /bin/sh
+# Copyright (C) 2012 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Check that, if we have two pattern rules from which the same file (or
+# set of files) can be built, and both are applicable, the first one wins.
+# This is used at least in our 'parallel-tests' support.
+
+am_create_testdir=empty
+. ./defs || Exit 1
+
+cat > Makefile <<'END'
+default:
+
+%.foo: %
+ cp $< $@
+%.foo: %.x
+ cp $< $@
+
+%.bar: %.x
+ cp $< $@
+%.bar: %
+
+%.mu %.fu: %.1
+ cp $< $*.mu && cp $< $*.fu
+%.mu %.fu: %.2
+ cp $< $*.mu && cp $< $*.fu
+END
+
+echo one > all
+echo two > all.x
+$MAKE all.foo all.bar
+diff all all.foo
+diff all.x all.bar
+
+echo one > x.1
+echo two > x.2
+$MAKE x.mu
+diff x.mu x.1
+diff x.fu x.1
+
+:
diff --git a/t/tap-bad-prog.tap b/t/tap-bad-prog.tap
index 580e7d3..5eb1c56 100755
--- a/t/tap-bad-prog.tap
+++ b/t/tap-bad-prog.tap
@@ -30,8 +30,9 @@ AC_OUTPUT
END
cat > Makefile.am << 'END'
-TEST_LOG_DRIVER = $(srcdir)/tap-driver
-TESTS = none.test noread.test noexec.test
+LOG_DRIVER = $(srcdir)/tap-driver
+TEST_LOG_DRIVER = $(LOG_DRIVER)
+TESTS = none.test noread noexec.test
none.test:
END
@@ -41,8 +42,8 @@ echo 1..1
echo ok 1
END
-cp noexec.test noread.test
-chmod a-r noread.test
+cp noexec.test noread
+chmod a-r noread
$ACLOCAL
$AUTOCONF
@@ -72,10 +73,10 @@ else
fi
desc="non-readable test is reported"
-if test -r noread.test; then
+if test -r noread; then
skip_ -r "any file is readable" "$desc"
else
- command_ok_ "$desc" -- grep '^ERROR: noread\.test' stdout
+ command_ok_ "$desc" -- grep '^ERROR: noread' stdout
fi
# Check that no spurious test result is reported. This is lower-priority
diff --git a/t/tap-basic.sh b/t/tap-basic.sh
index 96578e5..417c248 100755
--- a/t/tap-basic.sh
+++ b/t/tap-basic.sh
@@ -47,6 +47,8 @@ ok.test:
echo 'ok 3 # SKIP' >>address@hidden
cat address@hidden ;: For debugging.
mv -f address@hidden $@
+
+LOG_DRIVER = false # Dummy but required.
END
cat > success.test << 'END'
diff --git a/t/tap-common-setup.sh b/t/tap-common-setup.sh
index b14099a..cd0a14d 100755
--- a/t/tap-common-setup.sh
+++ b/t/tap-common-setup.sh
@@ -19,7 +19,12 @@
am_parallel_tests=yes
. ./defs || Exit 1
-cat >> configure.ac << END
+cat >> configure.ac << 'END'
+# FIXME: must define this otherwise automake will require the presence
+# FIXME: of the 'test-driver' script. This issue should be documented
+# FIXME: in the manual ...
+AC_SUBST([LOG_DRIVER],
+ ['$(error LOG_DRIVER should be never used) false'])
AC_OUTPUT
END
@@ -32,6 +37,7 @@ END
$ACLOCAL
$AUTOCONF
$AUTOMAKE
+test ! -f test-driver
./configure
diff --git a/t/tap-diagnostic-custom.sh b/t/tap-diagnostic-custom.sh
index d3f28a5..bb189fd 100755
--- a/t/tap-diagnostic-custom.sh
+++ b/t/tap-diagnostic-custom.sh
@@ -32,6 +32,7 @@ my_log_driver = $(srcdir)/tap-driver
my_log_compiler = cat
TEST_EXTENSIONS =
TESTS =
+LOG_DRIVER = $(error LOG_DRIVER is dummy, required, never extended)
END
: > later.mk
diff --git a/t/tap-more.sh b/t/tap-more.sh
index 00abbf2..14181b6 100755
--- a/t/tap-more.sh
+++ b/t/tap-more.sh
@@ -29,7 +29,8 @@ am_parallel_tests=yes
fetch_tap_driver
-cat >> configure.ac <<END
+cat >> configure.ac << 'END'
+AC_SUBST([LOG_DRIVER], ["dummy but required"])
AC_SUBST([AM_TEST_LOG_DRIVER_FLAGS], ['--comments'])
AC_OUTPUT
END
diff --git a/t/tap-recheck.sh b/t/tap-recheck.sh
index 4b120a5..a2af067 100755
--- a/t/tap-recheck.sh
+++ b/t/tap-recheck.sh
@@ -22,11 +22,12 @@ am_parallel_tests=yes
fetch_tap_driver
-cat >> configure.ac <<END
+cat >> configure.ac << 'END'
AC_OUTPUT
END
cat > Makefile.am << 'END'
+LOG_DRIVER = $(error LOG_DRIVER should never be expanded) false
TEST_LOG_DRIVER = $(srcdir)/tap-driver
TESTS = a.test b.test c.test d.test
END
diff --git a/t/tap-signal.tap b/t/tap-signal.tap
index ba17bd4..867b762 100755
--- a/t/tap-signal.tap
+++ b/t/tap-signal.tap
@@ -67,7 +67,7 @@ chmod a+x *.test
$ACLOCAL
$AUTOCONF
-$AUTOMAKE
+$AUTOMAKE -a
./configure
diff --git a/t/test-driver-custom-multitest-recheck.sh
b/t/test-driver-custom-multitest-recheck.sh
index 9b4f5df..af3d851 100755
--- a/t/test-driver-custom-multitest-recheck.sh
+++ b/t/test-driver-custom-multitest-recheck.sh
@@ -82,7 +82,7 @@ chmod a+x *.test
$ACLOCAL
$AUTOCONF
-$AUTOMAKE
+$AUTOMAKE -a
do_recheck ()
{
diff --git a/t/test-driver-custom-multitest-recheck2.sh
b/t/test-driver-custom-multitest-recheck2.sh
index cf97135..eef1043 100755
--- a/t/test-driver-custom-multitest-recheck2.sh
+++ b/t/test-driver-custom-multitest-recheck2.sh
@@ -70,7 +70,7 @@ chmod a+x *.test
$ACLOCAL
$AUTOCONF
-$AUTOMAKE
+$AUTOMAKE -a
for vpath in : false; do
if $vpath; then
diff --git a/t/test-driver-custom-multitest.sh
b/t/test-driver-custom-multitest.sh
index d5e2cf5..caf6a07 100755
--- a/t/test-driver-custom-multitest.sh
+++ b/t/test-driver-custom-multitest.sh
@@ -30,6 +30,8 @@ AC_OUTPUT
END
cat > Makefile.am << 'END'
+LOG_DRIVER = dummy but required
+
TEST_EXTENSIONS = .t
T_LOG_DRIVER = $(SHELL) $(srcdir)/trivial-test-driver
diff --git a/t/test-driver-custom-no-extra-driver.sh
b/t/test-driver-custom-no-extra-driver.sh
index 31000af..5f66f6d 100755
--- a/t/test-driver-custom-no-extra-driver.sh
+++ b/t/test-driver-custom-no-extra-driver.sh
@@ -24,7 +24,7 @@ am_parallel_tests=yes
cat >> configure.ac <<'END'
AC_PROG_CC
AC_CONFIG_FILES([sub1/Makefile sub2/Makefile])
-AC_SUBST([X_LOG_DRIVER], [none])
+AC_SUBST([LOG_DRIVER], ['unused but required by automake'])
AC_OUTPUT
END
@@ -32,7 +32,6 @@ mkdir sub1 sub2
cat > Makefile.am <<'END'
SUBDIRS = sub1 sub2
-LOG_DRIVER =
TEST_LOG_DRIVER = :
TESTS = foo bar.test
END
@@ -41,12 +40,12 @@ cat > sub1/Makefile.am <<'END'
TEST_EXTENSIONS = .x .sh .pl
SH_LOG_DRIVER = dummy1
PL_LOG_DRIVER = dummy2
+X_LOG_DRIVER = dummy3
TESTS = a.pl b.sh c.x
END
cat > sub2/Makefile.am <<'END'
TEST_EXTENSIONS = .bar
-LOG_DRIVER = x
BAR_LOG_DRIVER = y
TESTS = 1 2.bar 3.test 4.t 5.tt $(check_PROGRAMS)
check_PROGRAMS = p1 p2$(EXEEXT) p3.bar p4.suf
diff --git a/t/test-driver-custom-xfail-tests.sh
b/t/test-driver-custom-xfail-tests.sh
index 745772f..d883b73 100755
--- a/t/test-driver-custom-xfail-tests.sh
+++ b/t/test-driver-custom-xfail-tests.sh
@@ -136,7 +136,7 @@ chmod a+x td
$ACLOCAL
$AUTOCONF
-$AUTOMAKE
+$AUTOMAKE --add-missing
./configure
diff --git a/t/test-driver-fail.sh b/t/test-driver-fail.sh
index 0eb2268..f1c3df8 100755
--- a/t/test-driver-fail.sh
+++ b/t/test-driver-fail.sh
@@ -30,14 +30,16 @@ AC_OUTPUT
END
cat > Makefile.am <<'END'
-TEST_LOG_DRIVER = ./oops
-TESTS = foo.test
+TEST_EXTENSIONS =
+LOG_DRIVER = ./oops
+TESTS = foo
END
-cat > foo.test <<'END'
+cat > foo <<'END'
#! /bin/sh
exit 0
END
+chmod a+x foo
$ACLOCAL
$AUTOCONF
diff --git a/t/test-driver-strip-vpath.sh b/t/test-driver-strip-vpath.sh
index 9670188..f7adb67 100755
--- a/t/test-driver-strip-vpath.sh
+++ b/t/test-driver-strip-vpath.sh
@@ -39,9 +39,10 @@ cat > Makefile.am << 'END'
# The $(empty) are for eliciting VPATH rewrites on make implementations
# that support it (e.g., Solaris make), to improve coverage.
empty =
-TESTS = $(empty) foo.test src/bar.test ./src/baz.test $(empty)
+TESTS = $(empty) foo.test src/bar.test ./src/baz $(empty)
$(TESTS):
-TEST_LOG_DRIVER = $(srcdir)/checkstrip-driver
+LOG_DRIVER = $(srcdir)/checkstrip-driver
+TEST_LOG_DRIVER = $(LOG_DRIVER)
EXTRA_DIST = checkstrip-driver
END
@@ -61,8 +62,10 @@ while test $# -gt 0; do
done
echo "test name: $test_name" # For debugging.
case $test_name in
- foo.test|./foo.test|src/ba[rz].test|./src/ba[rz].test);;
- *) exit 1;;
+ foo.test|./foo.test) ;;
+ src/bar.test|./src/bar.test) ;;
+ src/baz|./src/baz) ;;
+ *) exit 1 ;;
esac
echo dummy > "$log_file"
echo dummy > "$trs_file"
diff --git a/t/test-metadata-global-log.sh b/t/test-metadata-global-log.sh
index b10f481..5d02e77 100755
--- a/t/test-metadata-global-log.sh
+++ b/t/test-metadata-global-log.sh
@@ -24,6 +24,7 @@ am_parallel_tests=yes
. ./defs || Exit 1
cat >> configure.ac << 'END'
+AC_SUBST([LOG_DRIVER], ['ignored but required by autoamke, sigh!'])
AC_OUTPUT
END
diff --git a/t/test-metadata-global-result.sh b/t/test-metadata-global-result.sh
index b247444..9c0fd48 100755
--- a/t/test-metadata-global-result.sh
+++ b/t/test-metadata-global-result.sh
@@ -29,7 +29,8 @@ cat > Makefile.am << 'END'
TEST_EXTENSIONS = .test .x
TEST_LOG_DRIVER = ./dummy-driver
X_LOG_DRIVER = ./dummy-driver
-TESTS = foo.test zar-doz.test
+LOG_DRIVER = ./dummy-driver
+TESTS = foo.test zar-doz.test mu
END
cat > dummy-driver <<'END'
@@ -76,6 +77,7 @@ $AUTOMAKE
: Basic checks.
echo :global-test-result: PASS > foo.test
+echo :global-test-result: PASS > mu
echo :global-test-result: ERROR > zar-doz.x
$MAKE check
@@ -87,6 +89,11 @@ PASS: foo
END
have_result <<END
+PASS: mu
+========
+END
+
+have_result <<END
ERROR: zar-doz
==============
END
@@ -95,6 +102,7 @@ END
echo :global-test-result: PASS/SKIP > foo.test
echo :global-test-result: ALMOST PASSED > zar-doz.x
+echo :global-test-result: HU? > mu
$MAKE check
cat test-suite.log
@@ -109,6 +117,11 @@ ALMOST PASSED: zar-doz
======================
END
+have_result <<END
+HU?: mu
+=======
+END
+
: Fields ':test-result:' does not interfere with the global test result.
cat > foo.test << 'END'
@@ -123,6 +136,13 @@ cat > zar-doz.x << 'END'
:test-result: XFAIL
END
+cat > mu << 'END'
+:test-result: PASS
+:global-test-result: XFAIL
+:test-result: PASS
+:test-result: PASS
+END
+
$MAKE check && Exit 1
cat test-suite.log
@@ -136,6 +156,11 @@ FAIL: zar-doz
=============
END
+have_result <<END
+XFAIL: mu
+=========
+END
+
: What happens when ':global-test-result:' is absent.
cat > foo.test << 'END'
@@ -143,6 +168,7 @@ cat > foo.test << 'END'
:test-result: ERROR
END
: > zar-doz.x
+: > mu
$MAKE check && Exit 1
cat test-suite.log
@@ -157,10 +183,16 @@ RUN: zar-doz
============
END
+have_result <<END
+RUN: mu
+=======
+END
+
# Leading and trailing whitespace gets eaten/normalized.
echo ":global-test-result:SKIP${tab} ${tab}${tab}" > foo.test
echo ":global-test-result:${tab} ${tab}XFAIL ${tab} " > zar-doz.x
+echo ":global-test-result: ${tab} PASS${tab} ${tab}${tab}" > mu
$MAKE check
cat test-suite.log
@@ -175,10 +207,16 @@ XFAIL: zar-doz
==============
END
+have_result <<END
+PASS: mu
+========
+END
+
# Whitespaces before and after ':global-test-result:' are handled OK.
echo " $tab:global-test-result:PASS" > foo.test
echo "${tab}${tab}:global-test-result:${tab} ${tab}SKIP" > zar-doz.x
+echo " :global-test-result:${tab}SKIP" > mu
$MAKE check
cat test-suite.log
@@ -193,4 +231,9 @@ SKIP: zar-doz
=============
END
+have_result <<END
+SKIP: mu
+========
+END
+
:
diff --git a/t/test-metadata-results.sh b/t/test-metadata-results.sh
index 5978f7f..1cafbb5 100755
--- a/t/test-metadata-results.sh
+++ b/t/test-metadata-results.sh
@@ -25,8 +25,9 @@ AC_OUTPUT
END
cat > Makefile.am << 'END'
-TEST_LOG_DRIVER = ./dummy-driver
-TESTS = foo.test bar.test
+LOG_DRIVER = ./dummy-driver
+TEST_LOG_DRIVER = ${LOG_DRIVER}
+TESTS = foo.test bar
END
cat > dummy-driver <<'END'
@@ -88,7 +89,7 @@ $AUTOMAKE
# "old-style" directives with format "RESULT: test-name" are now ignored.
: > foo.test
-echo blah blah blah > bar.test
+echo blah blah blah > bar
mk_check
count_test_results total=0 pass=0 fail=0 xpass=0 xfail=0 skip=0 error=0
@@ -96,7 +97,7 @@ cat > foo.test <<END
:test-global-result: PASS
:test-result: FAIL
END
-cat > bar.test <<END
+cat > bar <<END
:test-result: SKIP
:test-global-result: ERROR
END
@@ -108,7 +109,7 @@ FAIL: foo.test
:test-result: PASS
:test-global-result: XPASS
END
-echo ERROR: bar.test > bar.test
+echo ERROR: bar > bar
mk_check
count_test_results total=1 pass=1 fail=0 xpass=0 xfail=0 skip=0 error=0
@@ -116,7 +117,7 @@ cat > foo.test <<END
:test-global-result: SKIP
:test-result: FAIL
END
-cat > bar.test <<END
+cat > bar <<END
:test-global-result: PASS
END
mk_check && Exit 1
@@ -127,7 +128,7 @@ cat > foo.test <<END
:test-result: PASS
:test-result: SKIP
END
-cat > bar.test <<END
+cat > bar <<END
:test-result: SKIP
:test-result: PASS
:test-result: SKIP
@@ -147,18 +148,18 @@ cat > foo.test <<END
:test-result: XPASS
:test-result: ERROR
END
-: > bar.test
+: > bar
mk_check && Exit 1
count_test_results total=6 pass=1 fail=1 xpass=1 xfail=1 skip=1 error=1
-cp foo.test bar.test
+cp foo.test bar
mk_check && Exit 1
count_test_results total=12 pass=2 fail=2 xpass=2 xfail=2 skip=2 error=2
# Check that we are liberal w.r.t. whitespace use.
: > foo.test
-: > bar.test
+: > bar
for RESULT in PASS FAIL XPASS XFAIL SKIP ERROR; do
sed -e 's/^ *//' -e 's/|//g' >> foo.test <<END
|:test-result:$RESULT|
@@ -166,10 +167,10 @@ for RESULT in PASS FAIL XPASS XFAIL SKIP ERROR; do
|:test-result:$RESULT $tab|
|:test-result:$tab$tab $RESULT$tab $tab |
END
- echo " $tab $tab$tab :test-result: $RESULT" >> bar.test
+ echo " $tab $tab$tab :test-result: $RESULT" >> bar
done
cat foo.test # For debugging.
-cat bar.test # Likewise.
+cat bar # Likewise.
mk_check && Exit 1
count_test_results total=30 pass=5 fail=5 xpass=5 xfail=5 skip=5 error=5
--
1.7.9.5
- [Automake-ng] [PATCH 00/10] Rewrite parallel-tests to rely on GNU make capabilities, Stefano Lattarini, 2012/04/23
- [Automake-ng] [PATCH 02/10] [ng] cleanup: simplify %transforms for parallel-tests handling, Stefano Lattarini, 2012/04/23
- [Automake-ng] [PATCH 05/10] [ng] parallel-tests: support "TESTS = $(srcdir)/foo.test", Stefano Lattarini, 2012/04/23
- [Automake-ng] [PATCH 04/10] [ng] coverage: parallel-tests and dynamic $(TESTS) content, Stefano Lattarini, 2012/04/23
- [Automake-ng] [PATCH 01/10] [ng] parallel-tests: simplify, using pattern rules with multiple targets, Stefano Lattarini, 2012/04/23
- [Automake-ng] [PATCH 03/10] [ng] parallel-tests: simplify automake-time preprocessing,
Stefano Lattarini <=
- [Automake-ng] [PATCH 07/10] [ng] tests: run long-running ones early (useful with parallel make), Stefano Lattarini, 2012/04/23
- [Automake-ng] [PATCH 06/10] [ng] build: define $(TESTS) through a wildcard, Stefano Lattarini, 2012/04/23
- [Automake-ng] [PATCH 08/10] [ng] parallel-tests: remove obsolete comment, Stefano Lattarini, 2012/04/23
- [Automake-ng] [PATCH 09/10] [ng] parallel-tests: fix confusing function names in Makefile recipes, Stefano Lattarini, 2012/04/23
- [Automake-ng] [PATCH 10/10] [ng] tests: fix a spurious failure on Solaris 10, Stefano Lattarini, 2012/04/23
- Re: [Automake-ng] [PATCH 00/10] Rewrite parallel-tests to rely on GNU make capabilities, Stefano Lattarini, 2012/04/26