[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Automake-ng] [PATCH 01/10] [ng] parallel-tests: simplify, using pattern
|
From: |
Stefano Lattarini |
|
Subject: |
[Automake-ng] [PATCH 01/10] [ng] parallel-tests: simplify, using pattern rules with multiple targets |
|
Date: |
Mon, 23 Apr 2012 11:57:19 +0200 |
In GNU make, a pattern with multiple targets, like:
%.tab.c %.tab.h: %.y
bison -d $<
informs make that the associated recipe (in this example, "bison -d foo.y")
will make create all the targets *at once* (in this example, both 'x.tab.c'
and 'x.tab.h').
Using this semantics we can simplify the parallel testsuite harness nicely,
especially the recipe to build 'test-suite.log' in face of partially
missing prerequisites (e.g., a missing '.trs' file while the corresponding
'.log' file is present and up-to-date).
In doing so, we unfortunately lose the ability to cope with very fringe-case
situations and weird errors (e.g., a '.log' or '.trs' file somehow created
unreadable), but that's a more than fair trade-off for the simplifications
we obtain.
* lib/am/check.am (am__set_TESTS_bases): Removed, superseded by ...
(am__TEST_BASES): ... this new variable.
(check-TESTS, recheck): Adjust these recipes.
(am__TEST_RESULTS): New variable.
(am__check_pre): Adjust, to avoid an extra error (syntax error in the
shell) when a circular dependency due to $(TEST_SUITE_LOG) being listed
in $(TEST_LOGS) is detected.
($(TEST_SUITE_LOG)): Recipe heavily edited and simplified. Also,
depend on $(am__TEST_RESULTS) explicitly.
(.log.trs): Remove this suffix rule, superseded by ...
* lib/am/check2.am (%.log %.trs): ... this new pattern rule.
Remove obsolete suffix rules.
(am__runtest): Adjust.
* automake.in (handle_tests): Adjust minimally.
* lib/Automake/Rule.pm: Skip detection of duplicated rules for pattern
rules; it would be tricky, prone to false positives, and not worth it.
* t/parallel-tests-fork-bomb.sh: Minimal adjustments.
* t/parallel-tests-unreadable.sh: Adjust functional tests.
* t/serial-tests.sh: Adjust grepping checks.
* t/serial-extensions.sh: Likewise.
* t/parallel-tests-fd-redirect.sh: Relax grepping checks.
* t/parallel-tests-fd-redirect-exeext.sh: Likewise.
* t/test-trs-basic.sh: Adjust checks about some internal details that
have been changed.
* t/parallel-tests-no-repeat.sh: New test.
* t/test-trs-recover-parallel.sh: Likewise.
* t/test-driver-trs-suffix-registered.sh: Remove as obsolete.
* t/test-missing2.sh: Remove (as "too picky").
* t/test-trs-recover2.sh: Likewise.
* t/list-of-tests.mk: Update.
Signed-off-by: Stefano Lattarini <address@hidden>
---
automake.in | 5 +-
lib/Automake/Rule.pm | 39 +++---
lib/am/check.am | 144 ++++++++------------
lib/am/check2.am | 13 +-
t/list-of-tests.mk | 5 +-
t/parallel-tests-exeext.sh | 3 -
t/parallel-tests-fd-redirect-exeext.sh | 7 -
t/parallel-tests-fd-redirect.sh | 6 -
t/parallel-tests-fork-bomb.sh | 3 +-
t/parallel-tests-no-repeat.sh | 44 ++++++
t/parallel-tests-unreadable.sh | 52 ++++---
t/serial-tests.sh | 4 +-
t/test-extensions.sh | 2 +-
t/test-missing2.sh | 54 --------
t/test-trs-basic.sh | 5 +-
...-registered.sh => test-trs-recover-parallel.sh} | 62 +++++----
t/test-trs-recover2.sh | 133 ------------------
17 files changed, 201 insertions(+), 380 deletions(-)
create mode 100755 t/parallel-tests-no-repeat.sh
delete mode 100755 t/test-missing2.sh
rename t/{test-driver-trs-suffix-registered.sh =>
test-trs-recover-parallel.sh} (52%)
delete mode 100755 t/test-trs-recover2.sh
diff --git a/automake.in b/automake.in
index 86da5a0..19c99c7 100644
--- a/automake.in
+++ b/automake.in
@@ -4716,7 +4716,6 @@ sub handle_per_suffix_test
{
$pfx = '';
$generic = 0;
- $am_exeext = 'FALSE';
}
else
{
@@ -4724,9 +4723,9 @@ sub handle_per_suffix_test
unless $test_suffix =~ m/^\.(.*)/;
$pfx = uc ($1) . '_';
$generic = 1;
- $am_exeext = exists $configure_vars{'EXEEXT'} ? 'am__EXEEXT'
- : 'FALSE';
}
+ $am_exeext = exists $configure_vars{'EXEEXT'} ? 'am__EXEEXT'
+ : 'FALSE';
# The "test driver" program, deputed to handle tests protocol used by
# test scripts. By default, it's assumed that no protocol is used,
# so we fall back to the old "parallel-tests" behaviour, implemented
diff --git a/lib/Automake/Rule.pm b/lib/Automake/Rule.pm
index f7bf545..27df7eb 100644
--- a/lib/Automake/Rule.pm
+++ b/lib/Automake/Rule.pm
@@ -618,23 +618,17 @@ sub _maybe_warn_about_duplicated_target ($$$$$$)
{
if ($oldowner == RULE_USER)
{
- # Ignore '%'-style pattern rules. We'd need the
- # dependencies to detect duplicates, and they are
- # already diagnosed as unportable by -Wportability.
- if ($target !~ /^[^%]*%[^%]*$/)
- {
- ## FIXME: Presently we can't diagnose duplicate user rules
- ## because we don't distinguish rules with commands
- ## from rules that only add dependencies. E.g.,
- ## .PHONY: foo
- ## .PHONY: bar
- ## is legitimate. (This is phony.test.)
-
- # msg ('syntax', $where,
- # "redefinition of '$target'$condmsg ...", partial => 1);
- # msg_cond_rule ('syntax', $cond, $target,
- # "... '$target' previously defined here");
- }
+ ## FIXME: Presently we can't diagnose duplicate user rules
+ ## because we don't distinguish rules with commands
+ ## from rules that only add dependencies. E.g.,
+ ## .PHONY: foo
+ ## .PHONY: bar
+ ## is legitimate. (This is phony.test.)
+
+ # msg ('syntax', $where,
+ # "redefinition of '$target'$condmsg ...", partial => 1);
+ # msg_cond_rule ('syntax', $cond, $target,
+ # "... '$target' previously defined here");
}
else
{
@@ -704,9 +698,13 @@ sub _conditionals_for_rule ($$$$)
return $cond if !$message; # No ambiguity.
+ # Ignore possible ambiguity in '%'-style pattern rules. We'd need the
+ # dependencies to detect duplicates, and would be overkill anyway, worth
+ # the possibility of annoying false positives.
+ return $cond if $target =~ /%/;
+
if ($owner == RULE_USER)
{
- # For user rules, just diagnose the ambiguity.
msg 'syntax', $where, "$message ...", partial => 1;
msg_cond_rule ('syntax', $ambig_cond, $target,
"... '$target' previously defined here");
@@ -774,7 +772,10 @@ sub define ($$$$$)
my $tdef = _rule_defn_with_exeext_awareness ($target, $cond, $where);
# See whether this is a duplicated target declaration.
- if ($tdef)
+ # Ignore '%'-style pattern rules. We'd need the dependencies to detect
+ # duplicates, and would be overkill anyway, worth the possibility of
+ # annoying false positives.
+ if ($tdef && $target !~ /%/)
{
# Diagnose invalid target redefinitions, if any. Note that some
# target redefinitions are valid (e.g., for multiple-targets
diff --git a/lib/am/check.am b/lib/am/check.am
index d470437..59c5829 100644
--- a/lib/am/check.am
+++ b/lib/am/check.am
@@ -82,7 +82,11 @@ srcdir=$(srcdir); export srcdir; \
test x$(@D) = x. || test -d $(@D) || $(MKDIR_P) $(@D) || exit $$?; \
f='$(patsubst $(srcdir)/%,%,$<)'; \
## We need to invoke the test in way that won't cause a PATH search.
-case $< in */*) tst=$<;; *) tst=./$<;; esac; \
+## Quotes around '$<' are required to avoid extra errors when a circular
+## dependency is detected (e.g., because $(TEST_SUITE_LOG) is in
+## $(TEST_LOGS)), because in that case '$<' expands to empty and an
+## unquote usage of it could cause syntax errors in the shell.
+case '$<' in */*) tst='$<';; *) tst=./'$<';; esac; \
if test -n '$(DISABLE_HARD_ERRORS)'; then \
am__enable_hard_errors=no; \
else \
@@ -94,88 +98,47 @@ case " $(strip $(XFAIL_TESTS)) " in \
esac; \
$(AM_TESTS_ENVIRONMENT) $(TESTS_ENVIRONMENT)
-# A shell command to get the names of the tests scripts with any registered
-# extension removed (i.e., equivalently, the names of the test logs, with
-# the '.log' extension removed). The result is saved in the shell variable
-# '$bases'. This honors runtime overriding of TESTS and TEST_LOGS.
-# It also works around the GNU make 3.80 bug where trailing whitespace in
-# "TESTS = foo.test $(empty)" causes $(TESTS_LOGS) to erroneously expand to
-# "foo.log .log".
-am__set_TESTS_bases = bases='$(strip $(TEST_LOGS:.log=))'
+# The names of the tests scripts with any registered extension removed
+# (i.e., equivalently, the names of the test logs, with the '.log' suffix
+# stripped), and the name of the test result files (i.e., equivalently, the
+# names of the test logs, with the '.log' suffix substituted by the '.trs'
+# suffix). This honors runtime overriding of TESTS and TEST_LOGS. It
+# also works around the GNU make 3.80 bug where trailing whitespace in
+# "TESTS = foo.test $(empty)" causes $(TESTS_LOGS) to erroneously expand
+# to "foo.log .log".
+am__TEST_BASES = $(patsubst %.log,%,$(strip $(TEST_LOGS)))
+am__TEST_RESULTS = $(patsubst %.log,%.trs,$(strip $(TEST_LOGS)))
-# Recover from deleted '.trs' file; this should ensure that
-# "rm -f foo.log; make foo.trs" re-run 'foo.test', and re-create
-# both 'foo.log' and 'foo.trs'. Break the recipe in two subshells
-# to avoid problems with "make -n".
-.log.trs:
- rm -f $< $@
- $(MAKE) $<
-
-$(TEST_SUITE_LOG): $(TEST_LOGS)
- @$(am__set_TESTS_bases); \
-## Helper shell function, tells whether a path refers to an existing,
-## regular, readable file.
- am__f_ok () { test -f "$$1" && test -r "$$1"; }; \
-## We need to ensures that all the required '.trs' and '.log' files will
-## be present and readable. The direct dependencies of $(TEST_SUITE_LOG)
-## only ensure that all the '.log' files exists; they don't ensure that
-## the '.log' files are readable, and worse, they don't ensure that the
-## '.trs' files even exist.
- redo_bases=`for i in $$bases; do \
- am__f_ok $$i.trs && am__f_ok $$i.log || echo $$i; \
- done`; \
- if test -n "$$redo_bases"; then \
-## Uh-oh, either some '.log' files were unreadable, or some '.trs' files
-## were missing (or unreadable). We need to re-run the corresponding
-## tests in order to re-create them.
- redo_logs=`for i in $$redo_bases; do echo $$i.log; done`; \
- redo_results=`for i in $$redo_bases; do echo $$i.trs; done`; \
- if $(am__make_dryrun); then :; else \
-## Break "rm -f" into two calls to minimize the possibility of exceeding
-## command line length limits.
- rm -f $$redo_logs && rm -f $$redo_results || exit 1; \
- fi; \
- fi; \
-## Use a trick to to ensure that we don't go into an infinite recursion
-## in case a test log in $(TEST_LOGS) is the same as $(TEST_SUITE_LOG).
-## Yes, this has already happened in practice. Sigh!
- if test -n "$$am__remaking_logs"; then \
- echo "fatal: making $(TEST_SUITE_LOG): possible infinite" \
- "recursion detected" >&2; \
- else \
- am__remaking_logs=yes $(MAKE) $$redo_logs; \
- fi; \
- if $(am__make_dryrun); then :; else \
-## Sanity check: each unreadable or non-existent test result file should
-## has been properly remade at this point, as should the corresponding log
-## file.
- st=0; \
- errmsg="fatal: making $(TEST_SUITE_LOG): failed to create"; \
- for i in $$redo_bases; do \
- test -f $$i.trs && test -r $$i.trs \
- || { echo "$$errmsg $$i.trs" >&2; st=1; }; \
- test -f $$i.log && test -r $$i.log \
- || { echo "$$errmsg $$i.log" >&2; st=1; }; \
- done; \
- test $$st -eq 0 || exit 1; \
- fi
-## We need a new subshell to work portably with "make -n", since the
-## previous part of the recipe contained a $(MAKE) invocation.
- @$(am__sh_e_setup); $(am__tty_colors); $(am__set_TESTS_bases); \
+$(TEST_SUITE_LOG): $(TEST_LOGS) $(am__TEST_RESULTS)
+ @$(am__sh_e_setup); $(am__tty_colors); \
+ fatal () { echo "fatal: making $@: $$*" >&2; exit 1; }; \
+## Detect a possible circular dependency, and error out.
+ case ' $(strip $(TEST_LOGS)) ' in *' $(TEST_SUITE_LOG) '*) \
+ fatal "depends on itself (check TESTS content)";; \
+ esac; \
ws='[ ]'; \
-## List of test result files.
- results=`for b in $$bases; do echo $$b.trs; done`; \
- test -n "$$results" || results=/dev/null; \
+ count_result () \
+ { \
+ r='$(strip $(am__TEST_RESULTS))'; \
+ r=`grep "^$$ws*:test-result:$$ws*$${1-}" $$r </dev/null`; \
+## Catch I/O errors, and complain accordingly.
+ test $$? -le 1 || fatal "I/O error reading test results"; \
+ case $$r in \
+ "") echo 0;; \
+ * ) printf '%s\n' "$$r" | wc -l;; \
+ esac; \
+ }; \
## Prepare data for the test suite summary. These do not take into account
## unreadable test results, but they'll be appropriately updated later if
## needed.
- all=` grep "^$$ws*:test-result:" $$results | wc -l`; \
- pass=` grep "^$$ws*:test-result:$$ws*PASS" $$results | wc -l`; \
- fail=` grep "^$$ws*:test-result:$$ws*FAIL" $$results | wc -l`; \
- skip=` grep "^$$ws*:test-result:$$ws*SKIP" $$results | wc -l`; \
- xfail=`grep "^$$ws*:test-result:$$ws*XFAIL" $$results | wc -l`; \
- xpass=`grep "^$$ws*:test-result:$$ws*XPASS" $$results | wc -l`; \
- error=`grep "^$$ws*:test-result:$$ws*ERROR" $$results | wc -l`; \
+ all=`count_result` \
+ && pass=` count_result PASS` \
+ && fail=` count_result FAIL` \
+ && skip=` count_result SKIP` \
+ && xfail=`count_result XFAIL` \
+ && xpass=`count_result XPASS` \
+ && error=`count_result ERROR` \
+ || exit 1; \
## Whether the testsuite was successful or not.
if test `expr $$fail + $$xpass + $$error` -eq 0; then \
success=true; \
@@ -220,14 +183,15 @@ $(TEST_SUITE_LOG): $(TEST_LOGS)
result_count $$1 "ERROR:" $$error "$$mgn"; \
}; \
## Write "global" testsuite log.
- { \
+ if { \
+ st=0; \
echo "$(PACKAGE_STRING): $(subdir)/$(TEST_SUITE_LOG)" | \
$(am__rst_title); \
create_testsuite_report --no-color; \
echo; \
echo ".. contents:: :depth: 2"; \
echo; \
- for i in $$bases; do \
+ bases='$(am__TEST_BASES)'; for i in $$bases; do \
## FIXME: one fork per test -- this is horrendously inefficient!
if grep "^$$ws*:copy-in-global-log:$$ws*no$$ws*$$" $$i.trs \
>/dev/null; then continue; \
@@ -242,15 +206,17 @@ $(TEST_SUITE_LOG): $(TEST_LOGS)
test -n "$$glob_res" || glob_res=RUN; \
## Write the name and result of the test as an RST section title.
echo "$$glob_res: $$i" | $(am__rst_section); \
-## If we should have remade any unreadable '.log', above.
- if test ! -r $$i.log; then \
- echo "fatal: making $@: $$i.log is unreadable" >&2; \
- exit 1; \
- fi; \
- cat $$i.log; echo; \
+## Register any failure in reading test logs, to report an error later.
+ cat $$i.log || st=1; \
+ echo; \
done; \
- } >$(TEST_SUITE_LOG).tmp; \
- mv $(TEST_SUITE_LOG).tmp $(TEST_SUITE_LOG); \
+ test $$st -eq 0; \
+ } >$(TEST_SUITE_LOG).tmp; then \
+ mv -f $(TEST_SUITE_LOG).tmp $(TEST_SUITE_LOG); \
+ else \
+ rm -f $(TEST_SUITE_LOG).tmp; \
+ fatal "I/O error reading test logs"; \
+ fi; \
## Emit the test summary on the console.
if $$success; then \
col="$$grn"; \
@@ -310,7 +276,7 @@ check-TESTS recheck:
## we rely on .PHONY to work portably.
@test -z "$(TEST_SUITE_LOG)" || rm -f $(TEST_SUITE_LOG)
@ws='[ ]'; \
- log_list='' trs_list=''; $(am__set_TESTS_bases); \
+ log_list='' trs_list=''; bases='$(am__TEST_BASES)'; \
for i in $$bases; do \
## If running a "make recheck", we must only consider tests that had an
## unexpected outcome (FAIL or XPASS) in the earlier run. In particular,
diff --git a/lib/am/check2.am b/lib/am/check2.am
index fce5498..19aef63 100644
--- a/lib/am/check2.am
+++ b/lib/am/check2.am
@@ -19,8 +19,8 @@ am__runtest = \
$(am__check_pre) \
$($(1)LOG_DRIVER) \
--test-name "$$f" \
- --log-file $(2).log \
- --trs-file $(2).trs \
+ --log-file $*.log \
+ --trs-file $*.trs \
$(am__common_driver_flags) \
$(AM_$(1)LOG_DRIVER_FLAGS) \
$($(1)LOG_DRIVER_FLAGS) \
@@ -33,15 +33,14 @@ am__runtest = \
endif %?FIRST%
## From a test file to a .log and .trs file.
-?GENERIC?%EXT%.log:
-?!GENERIC?%OBJ%: %SOURCE%
- @$(call am__runtest,%PFX%,%BASE%)
+%.log %.trs: %%EXT%
+ @$(call am__runtest,%PFX%)
## If no programs are built in this package, then this rule is removed
## at automake time. Otherwise, %am__EXEEXT% expands to a configure time
## conditional, true if $(EXEEXT) is nonempty, thus this rule does not
## conflict with the previous one.
if %am__EXEEXT%
-?GENERIC?%EXT%$(EXEEXT).log:
- @$(call am__runtest,%PFX%,%BASE%)
+%.log %.trs: %%EXT%$(EXEEXT)
+ @$(call am__runtest,%PFX%)
endif %am__EXEEXT%
diff --git a/t/list-of-tests.mk b/t/list-of-tests.mk
index 5c67db5..436e401 100644
--- a/t/list-of-tests.mk
+++ b/t/list-of-tests.mk
@@ -737,6 +737,7 @@ t/parallel-tests6.sh \
t/parallel-tests8.sh \
t/parallel-tests9.sh \
t/parallel-tests10.sh \
+t/parallel-tests-no-repeat.sh \
t/parallel-tests-recheck-depends-on-all.sh \
t/parallel-tests-exeext.sh \
t/parallel-tests-suffix.sh \
@@ -785,7 +786,6 @@ t/test-driver-custom-multitest-recheck.sh \
t/test-driver-custom-multitest-recheck2.sh \
t/test-driver-create-log-dir.sh \
t/test-driver-strip-vpath.sh \
-t/test-driver-trs-suffix-registered.sh \
t/test-driver-fail.sh \
t/test-driver-is-distributed.sh \
t/test-harness-vpath-rewrite.sh \
@@ -796,10 +796,9 @@ t/test-metadata-global-result.sh \
t/test-metadata-recheck.sh \
t/test-metadata-results.sh \
t/test-missing.sh \
-t/test-missing2.sh \
t/test-trs-basic.sh \
t/test-trs-recover.sh \
-t/test-trs-recover2.sh \
+t/test-trs-recover-parallel.sh \
t/test-extensions.sh \
t/test-extensions-cond.sh \
t/parse.sh \
diff --git a/t/parallel-tests-exeext.sh b/t/parallel-tests-exeext.sh
index e93f5f1..ac94533 100755
--- a/t/parallel-tests-exeext.sh
+++ b/t/parallel-tests-exeext.sh
@@ -61,7 +61,4 @@ test ! -r y.bin.log
test -f b.log
test ! -r b.test.log
-$EGREP '^y\.log: y(\$\(EXEEXT\)|\.bin)' Makefile
-$EGREP '^\.test(\$\(EXEEXT\)|\.bin)\.log:' Makefile
-
:
diff --git a/t/parallel-tests-fd-redirect-exeext.sh
b/t/parallel-tests-fd-redirect-exeext.sh
index 56d7f59..7904b2d 100755
--- a/t/parallel-tests-fd-redirect-exeext.sh
+++ b/t/parallel-tests-fd-redirect-exeext.sh
@@ -98,13 +98,6 @@ else
fatal_ "unexpected error in ./configure"
fi
-# Sanity checks.
-st=0
-grep '^baz\.log:.*baz\$(EXEEXT)' Makefile || st=1
-grep '^\.test\$(EXEEXT)\.log:' Makefile || st=1
-grep '^qux\.log:' Makefile && st=1
-test $st -eq 0 || fatal_ "doesn't cover expected code paths"
-
st=0
$MAKE check >stdout || st=1
cat stdout
diff --git a/t/parallel-tests-fd-redirect.sh b/t/parallel-tests-fd-redirect.sh
index cf7acce..ad5815d 100755
--- a/t/parallel-tests-fd-redirect.sh
+++ b/t/parallel-tests-fd-redirect.sh
@@ -52,12 +52,6 @@ chmod a+x foo.sh bar
./configure
-# Sanity checks.
-st=0
-grep '^bar\.log:.*bar' Makefile || st=1
-grep '^foo\.log:' Makefile && st=1
-test $st -eq 0 || fatal_ "doesn't cover expected code paths"
-
st=0; $MAKE check >stdout || st=1
cat stdout
cat foo.log
diff --git a/t/parallel-tests-fork-bomb.sh b/t/parallel-tests-fork-bomb.sh
index 4daa811..193a6e7 100755
--- a/t/parallel-tests-fork-bomb.sh
+++ b/t/parallel-tests-fork-bomb.sh
@@ -61,9 +61,8 @@ do_check ()
$MAKE "$@" check >output 2>&1 || st=$?
cat output
$FGREP '::OOPS::' output && Exit 1 # Possible infinite recursion.
- # Check that at least we don't create a botched global log file.
- test ! -f "$log"
grep "[Cc]ircular.*dependency" output | $FGREP "$log"
+ grep "$log:.*depends on itself" output
test $st -gt 0
}
diff --git a/t/parallel-tests-no-repeat.sh b/t/parallel-tests-no-repeat.sh
new file mode 100755
index 0000000..c3220c0
--- /dev/null
+++ b/t/parallel-tests-no-repeat.sh
@@ -0,0 +1,44 @@
+#! /bin/sh
+# Copyright (C) 2012 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# The parallel-tests harness do not cause the same test to be
+# uselessly run multiple times.
+
+am_parallel_tests=yes
+. ./defs || Exit 1
+
+echo AC_OUTPUT >> configure.ac
+echo TESTS = foo.test > Makefile.am
+
+cat > foo.test <<'END'
+#! /bin/sh
+ls -l && mkdir bar
+END
+chmod a+x foo.test
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+
+./configure
+
+$MAKE -j1 check || { cat test-suite.log; Exit 1; }
+rmdir bar
+$MAKE -j2 check || { cat test-suite.log; Exit 1; }
+rmdir bar
+$MAKE -j4 check || { cat test-suite.log; Exit 1; }
+
+:
diff --git a/t/parallel-tests-unreadable.sh b/t/parallel-tests-unreadable.sh
index 7ddb9db..a3380c3 100755
--- a/t/parallel-tests-unreadable.sh
+++ b/t/parallel-tests-unreadable.sh
@@ -34,13 +34,11 @@ END
cat > foo.test << 'END'
#! /bin/sh
-echo foofoofoo
exit 0
END
cat > bar.test << 'END'
#! /bin/sh
-echo barbarbar
exit 77
END
@@ -52,25 +50,41 @@ $AUTOMAKE -a
./configure
-for files in \
- 'foo.log bar.log' \
- 'foo.trs bar.trs' \
- 'foo.trs bar.log' \
- 'foo.log bar.trs' \
-; do
+doit ()
+{
+ rm -f $*
$MAKE check
rm -f test-suite.log
- chmod a-r $files
- $MAKE test-suite.log || { ls -l; Exit 1; }
- ls -l
- grep '^foofoofoo$' foo.log
- grep '^:test-result: PASS' foo.trs
- grep '^barbarbar$' bar.log
- grep '^:test-result: SKIP' bar.trs
- grep '^SKIP: bar' test-suite.log
- grep '^barbarbar$' test-suite.log
- $EGREP ':.*foo|foofoofoo' test-suite.log && Exit 1
- : For shells with busted 'set -e'.
+ chmod a-r $*
+ $MAKE test-suite.log 2>stderr && { cat stderr >&2; Exit 1; }
+ cat stderr >&2
+}
+
+permission_denied ()
+{
+ # FIXME: there are systems where errors on permissions generate a
+ # FIXME: different message? We might experience spurious failures
+ # FIXME: there ...
+ grep "$1:.*[pP]ermission denied" stderr
+}
+
+for lst in bar.log 'foo.log bar.log'; do
+ doit $lst
+ permission_denied bar.log
+ grep 'test-suite\.log:.* I/O error reading test logs' stderr
done
+doit foo.trs
+permission_denied foo.trs
+grep 'test-suite\.log:.* I/O error reading test results' stderr
+
+doit foo.trs bar.trs
+permission_denied foo.trs
+permission_denied bar.trs
+grep 'test-suite\.log:.* I/O error reading test results' stderr
+
+doit foo.trs bar.log
+permission_denied foo.trs
+grep 'test-suite\.log:.* I/O error reading test results' stderr
+
:
diff --git a/t/serial-tests.sh b/t/serial-tests.sh
index 680513f..2661afa 100755
--- a/t/serial-tests.sh
+++ b/t/serial-tests.sh
@@ -32,8 +32,8 @@ has_parallel_tests ()
{
$EGREP '(^| )check-TESTS.*:' $1
$EGREP '(^| )recheck.*:' $1
- grep '^\$(TEST_SUITE_LOG): \$(TEST_LOGS)$' $1
- grep '^\.test\.log:$' $1
+ $EGREP '^\$\(TEST_SUITE_LOG\):.* \$\(TEST_LOGS\)( |$)' $1
+ grep '^%.log %.trs *:.*%\.test' $1
}
mkdir one two
diff --git a/t/test-extensions.sh b/t/test-extensions.sh
index c903798..13c42eb 100755
--- a/t/test-extensions.sh
+++ b/t/test-extensions.sh
@@ -42,7 +42,7 @@ grep -i 'log' Makefile.in # For debugging.
for lc in $valid_extensions; do
uc=`echo $lc | tr '[a-z]' '[A-Z]'`
grep "^${uc}_LOG_DRIVER =" Makefile.in
- grep "^\.${lc}\.log:" Makefile.in
+ grep "^%\.log %\.trs *:.*%\.${lc}" Makefile.in
done
# The produced Makefile is not broken.
diff --git a/t/test-missing2.sh b/t/test-missing2.sh
deleted file mode 100755
index dadc1b0..0000000
--- a/t/test-missing2.sh
+++ /dev/null
@@ -1,54 +0,0 @@
-#! /bin/sh
-# Copyright (C) 2011-2012 Free Software Foundation, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2, or (at your option)
-# any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with this program. If not, see <http://www.gnu.org/licenses/>.
-
-# parallel-tests:
-# - non-existent scripts listed in TESTS get diagnosed, even when
-# all the $(TEST_LOGS) have a dummy dependency.
-# See also related test 'test-missing.test'.
-
-am_parallel_tests=yes
-. ./defs || Exit 1
-
-cat >> configure.ac << 'END'
-AC_OUTPUT
-END
-
-cat > Makefile.am << 'END'
-TESTS = foobar1.test foobar2.test
-$(TEST_LOGS):
-END
-
-$ACLOCAL
-$AUTOCONF
-$AUTOMAKE -a
-
-./configure
-
-$MAKE foobar1.log foobar2.log || Exit 99
-test ! -f foobar1.log || Exit 99
-test ! -f foobar1.trs || Exit 99
-test ! -f foobar2.log || Exit 99
-test ! -f foobar2.trs || Exit 99
-
-$MAKE check 2>stderr && { cat stderr >&2; Exit 1; }
-cat stderr >&2
-grep 'test-suite\.log.*foobar1\.log' stderr
-grep 'test-suite\.log.*foobar1\.trs' stderr
-grep 'test-suite\.log.*foobar2\.log' stderr
-grep 'test-suite\.log.*foobar2\.trs' stderr
-test ! -f test-suite.log
-
-:
diff --git a/t/test-trs-basic.sh b/t/test-trs-basic.sh
index 55206e9..c988400 100755
--- a/t/test-trs-basic.sh
+++ b/t/test-trs-basic.sh
@@ -32,9 +32,10 @@ TEST_LOG_COMPILER = $(SHELL)
SH_LOG_COMPILER = $(SHELL)
## Used to check some internal details. And yes, the quotes around
-## '$bases' are deliberate: they check for whitespace normalization.
+## $(am__TESTS_BASES) are deliberate: they check for whitespace
+## normalization.
tb:
- $(am__set_TESTS_bases); echo "$$bases" > $@
+ echo '$(am__TEST_BASES)' > $@
END
cat > foo.test << 'END'
diff --git a/t/test-driver-trs-suffix-registered.sh
b/t/test-trs-recover-parallel.sh
similarity index 52%
rename from t/test-driver-trs-suffix-registered.sh
rename to t/test-trs-recover-parallel.sh
index 0c3734f..0a03bdf 100755
--- a/t/test-driver-trs-suffix-registered.sh
+++ b/t/test-trs-recover-parallel.sh
@@ -14,45 +14,47 @@
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
-# parallel-tests support: the following are registered with '.SUFFIXES':
-# - .log
-# - .trs (used by files that store test results and metadata)
-# - .test if $(TEST_EXTENSIONS) is not defined
-# - stuff in $(TEST_EXTENSIONS) otherwise
+# Check parallel harness features:
+# - recovery from deleted '.log' and '.trs' files, with parallel make
am_parallel_tests=yes
. ./defs || Exit 1
-: > Makefile.am
+all= log= trs=
+for i in 1 2 3 4 5 6 7 8 9 10 11 12; do
+ all="$all $i" log="$log $i" trs="$trs $i"
+done
-cat > 1.am << 'END'
-TESTS =
-END
+echo AC_OUTPUT >> configure.ac
+echo TESTS = > Makefile.am
-cat > 2.am << 'END'
-TEST_EXTENSIONS = .SH .abcdef
-TESTS =
-END
+for i in $all; do
+ echo TESTS += $i.test >> Makefile.am
+ (echo "#!/bin/sh" && echo "mkdir $i.d") > $i.test
+ chmod a+x $i.test
+done
-: > test-driver
+ls -l # For debugging.
$ACLOCAL
-
-$AUTOMAKE 1
-$AUTOMAKE 2
-
-sed -e 's/$/ /' 1.in > mk.1
-sed -e 's/$/ /' 2.in > mk.2
-
-grep '^\.SUFFIXES:' mk.1
-grep '^\.SUFFIXES:' mk.2
-
-for suf in test log trs; do
- grep "^\\.SUFFIXES:.* \\.$suf " mk.1
-done
-
-for suf in SH abcdef log trs; do
- grep "^\\.SUFFIXES:.* \\.$suf " mk.2
+$AUTOCONF
+$AUTOMAKE -a
+
+./configure
+
+: Create the required log files.
+$MAKE check
+
+for n in 1 2 5 7 12; do
+ for suf in log trs; do
+ rmdir *.d
+ rm -f *.$suf
+ $MAKE -j$n check
+ for f in $all; do
+ test -f $f.log
+ test -f $f.trs
+ done
+ done
done
:
diff --git a/t/test-trs-recover2.sh b/t/test-trs-recover2.sh
deleted file mode 100755
index ba59743..0000000
--- a/t/test-trs-recover2.sh
+++ /dev/null
@@ -1,133 +0,0 @@
-#! /bin/sh
-# Copyright (C) 2011-2012 Free Software Foundation, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2, or (at your option)
-# any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with this program. If not, see <http://www.gnu.org/licenses/>.
-
-# Check parallel harness features:
-# - recovery from unreadable '.trs' files, in various scenarios
-
-am_parallel_tests=yes
-. ./defs || Exit 1
-
-cat >> configure.ac <<END
-AC_OUTPUT
-END
-
-cat > Makefile.am << 'END'
-TESTS = foo.test bar.test
-TEST_LOG_COMPILER = true
-END
-
-: > foo.test
-: > bar.test
-
-TEST_STATUS=0; export TEST_STATUS
-
-$ACLOCAL
-$AUTOCONF
-$AUTOMAKE -a
-
-./configure
-
-: > t
-chmod a-r t
-test ! -r t || Exit 77
-rm -f t
-
-: Create the required log files.
-$MAKE check
-test -f foo.trs
-test -f bar.trs
-
-: Recreate with a sweeping "make check".
-chmod a-r bar.trs
-$MAKE check
-test -f foo.trs
-test -r foo.trs
-test -f bar.trs
-test -r bar.trs
-
-: Again, this time with one .trs file missing and the other
-: one unreadable.
-rm -f foo.trs
-chmod a-r bar.trs
-$MAKE check
-test -f foo.trs
-test -r foo.trs
-test -f bar.trs
-test -r bar.trs
-
-: Again, but using "make recheck" this time.
-rm -f foo.trs
-chmod a-r bar.trs
-$MAKE recheck >stdout || { cat stdout; Exit 1; }
-cat stdout
-test -f foo.trs
-test -r foo.trs
-test -f bar.trs
-test -r bar.trs
-grep '^PASS: foo\.test' stdout
-grep '^PASS: bar\.test' stdout
-
-: More complex interactions with "make recheck" are OK.
-chmod a-r bar.log bar.trs
-$MAKE recheck >stdout || { cat stdout; Exit 1; }
-cat stdout
-test -f bar.trs
-test -r bar.trs
-grep '^PASS: bar\.test' stdout
-grep 'foo\.test' stdout && Exit 1
-count_test_results total=1 pass=1 fail=0 xpass=0 xfail=0 skip=0 error=0
-
-: Recreate by remaking the global test log.
-chmod a-r foo.trs
-rm -f test-suite.log
-$MAKE test-suite.log >stdout || { cat stdout; Exit 1; }
-cat stdout
-test -f foo.trs
-test -r foo.trs
-grep '^PASS: foo\.test' stdout
-grep 'bar\.test' stdout && Exit 1
-# Also test that have only run before should be counted in the final
-# testsuite summary.
-grep '^# TOTAL: *2$' stdout
-
-: Setup for the next check.
-: > baz.test
-sed 's/^TESTS =.*/& baz.test/' Makefile > t
-diff t Makefile && Exit 99
-mv -f t Makefile
-$MAKE check
-test -f foo.trs
-test -f bar.trs
-test -f baz.trs
-
-: Interactions with lazy test reruns are OK.
-chmod a-r foo.trs
-$sleep
-touch stamp
-$sleep
-touch bar.test
-$MAKE RECHECK_LOGS= check >stdout || { cat stdout; Exit 1; }
-cat stdout
-test -r foo.trs
-is_newest bar.trs bar.test
-grep '^PASS: foo\.test' stdout
-grep '^PASS: bar\.test' stdout
-grep 'baz\.test' stdout && Exit 1
-# Also test that have only run before should be counted in the final
-# testsuite summary.
-grep '^# TOTAL: *3$' stdout
-
-:
--
1.7.9.5
- [Automake-ng] [PATCH 00/10] Rewrite parallel-tests to rely on GNU make capabilities, Stefano Lattarini, 2012/04/23
- [Automake-ng] [PATCH 02/10] [ng] cleanup: simplify %transforms for parallel-tests handling, Stefano Lattarini, 2012/04/23
- [Automake-ng] [PATCH 05/10] [ng] parallel-tests: support "TESTS = $(srcdir)/foo.test", Stefano Lattarini, 2012/04/23
- [Automake-ng] [PATCH 04/10] [ng] coverage: parallel-tests and dynamic $(TESTS) content, Stefano Lattarini, 2012/04/23
- [Automake-ng] [PATCH 01/10] [ng] parallel-tests: simplify, using pattern rules with multiple targets,
Stefano Lattarini <=
- [Automake-ng] [PATCH 03/10] [ng] parallel-tests: simplify automake-time preprocessing, Stefano Lattarini, 2012/04/23
- [Automake-ng] [PATCH 07/10] [ng] tests: run long-running ones early (useful with parallel make), Stefano Lattarini, 2012/04/23
- [Automake-ng] [PATCH 06/10] [ng] build: define $(TESTS) through a wildcard, Stefano Lattarini, 2012/04/23
- [Automake-ng] [PATCH 08/10] [ng] parallel-tests: remove obsolete comment, Stefano Lattarini, 2012/04/23
- [Automake-ng] [PATCH 09/10] [ng] parallel-tests: fix confusing function names in Makefile recipes, Stefano Lattarini, 2012/04/23
- [Automake-ng] [PATCH 10/10] [ng] tests: fix a spurious failure on Solaris 10, Stefano Lattarini, 2012/04/23
- Re: [Automake-ng] [PATCH 00/10] Rewrite parallel-tests to rely on GNU make capabilities, Stefano Lattarini, 2012/04/26