[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: CVS autoconf testsuite failure on Tru64 unix
From: |
Noah Misch |
Subject: |
Re: CVS autoconf testsuite failure on Tru64 unix |
Date: |
Sat, 18 Dec 2004 00:41:25 -0800 |
User-agent: |
Mutt/1.5.6i |
On Wed, Dec 15, 2004 at 02:19:48AM -0800, Noah Misch wrote:
> Now wait! Only two Autoconf tests use constructs that look like `${...}', and
> only one of those actually expands the parameter; the other is single-quoted
> and
> yields the syntax error that began this discussion. It would not be harmful
> to
> disable shell tracing for all such tests. Yes, that will be _so_ much
> cleaner.
> I'll post a patch separately. Thanks for not applying this one :-)
The natural fix involved moving much of the command analysis from runtime to
autom4te-time, so I did that. I am not certain that this patch is _simpler_
than the last patch, but I think each bit of complexity actually helps. The old
patch was complex in unhelpful ways.
Moving analysis to autom4te-time also shrunk `testsuite' from 2748 KiB to 2022
KiB. To be sure, I created that growth by adding all that shell code in the
first place. For full disclosure, these are the old and new `autom4te' run
times generating `testsuite':
With this patch: 12.70user 0.21system 0:13.50elapsed 95%CPU
Before: 12.17user 0.28system 0:13.34elapsed 93%CPU
The error from run to run is about +-.05user.
My post to address@hidden about expanding text for string functions concerned
this patch. An XFAIL test case demonstrates the lossage. I wanted to document
it also, but no good documentation words came to mind. If I think of some, I
will post them separately.
Thanks, Paul, for your help on this issue.
2004-12-18 Noah Misch <address@hidden>
* lib/autotest/general.m4 (_AT_DECIDE_TRACEABLE): New macro.
(_AT_CHECK): Use it.
* lib/m4sugar/m4sh.m4 (AS_ESCAPE_FOR_EXPAND): Remove.
(AS_ESCAPE): Fix comment.
* tests/autotest.at: Adjust section banner comments.
(AT_CHECK_AT): Accept STATUS and STDERR.
(AT_CHECK_AT_TEST): Likewise.
(Invalid brace-enclosed parameter expansion)
(Multiline command from M4 expansion)
(Double-M4-quoted command): New tests.
diff --exclude Makefile.am -urp -X dontdiff ac-clean/lib/autotest/general.m4
ac-simple_pexp/lib/autotest/general.m4
--- ac-clean/lib/autotest/general.m4 2004-12-14 22:30:02.000000000 -0500
+++ ac-simple_pexp/lib/autotest/general.m4 2004-12-18 02:16:29.284314403
-0500
@@ -1187,6 +1187,110 @@ m4_define([AT_CHECK_NOESCAPE],
[_AT_CHECK([$1],[$2],[$3],[$4],[$5],[$6])])
+# _AT_DECIDE_TRACEABLE(COMMANDS)
+# ------------------------------
+# Worker for for _AT_CHECK that expands to shell code. If COMMANDS are safe to
+# trace with `set -x', the shell code will set `at_trace_this=yes'. Otherwise,
+# the shell code will print a message stating an aspect of COMMANDS that makes
+# tracing them unsafe.
+#
+# Tracing COMMANDS is not safe if they contain a command that spans multiple
+# lines. When the test suite user passes `-x' or `--trace', the test suite
+# precedes every command with a `set -x'. Since most tests expect a specific
+# stderr, if only to confirm that it is empty, the test suite filters ^+ from
+# the captured stderr before comparing with the expected stderr. If a command
+# spans multiple lines, so will its trace, but a `+' only prefixes the first
+# line of that trace:
+#
+# $ echo 'foo
+# bar'
+# => stdout
+# foo
+# bar
+# => stderr
+# + foo
+# bar
+#
+# In a subset of cases, one could filter such extended shell traces from
stderr.
+# Since test commands spanning several lines are rare, I chose instead to
simply
+# not trace COMMANDS that could yield multiple trace lines. Distinguishing
such
+# COMMANDS became the task at hand.
+#
+# These features may cause a shell command to span multiple lines:
+#
+# (a) A quoted literal newline.
+# Example:
+# echo foo'
+# 'bar
+# M4 is a hostile language for the job of parsing COMMANDS to determine whether
+# each literal newline is quoted, so we simply disable tracing for all COMMANDS
+# that bear literal newlines.
+#
+# (b) A command substitution not subject to word splitting.
+# Example:
+# var=$(printf 'foo\nbar')
+# Example:
+# echo "`printf 'foo\\nbar`"
+# One cannot know in general the number of lines a command substitution will
+# yield without executing the substituted command. As such, we disable tracing
+# for all COMMANDS containing these constructs.
+#
+# (c) A parameter expansion not subject to word splitting.
+# Example:
+# var=foo'
+# 'bar
+# echo "$var"
+# Parameter expansions appear in COMMANDS with much greater frequency than do
+# newlines and command substitutions, so disabling tracing for all such
COMMANDS
+# would much more substantially devalue `testsuite -x'. To determine which
+# parameter expansions yield multiple lines, we escape all ``', `"', and `\' in
+# a copy of COMMANDS and expand that string within double quotes at runtime.
If
+# the result of that expansion contains multiple lines, the test suite disables
+# tracing for the command in question.
+#
+# This method leads the test suite to expand some parameters that the shell
+# itself will never expand due to single-quotes or backslash escapes. This is
+# not a problem for `$foo' expansions, which will simply yield the empty string
+# or some unrelated value. A `${...}' expansion could actually form invalid
+# shell code, however; consider `${=foo}'. Therefore, we disable tracing for
+# all COMMANDS containing `${...}'. This affects few COMMANDS.
+#
+# This macro falls in a very hot path; the Autoconf test suite expands it 1640
+# times as of this writing. To give a sense of the impact of the heuristics I
+# just described, the test suite preemptively disables tracing for 31 of those,
+# and 268 contain parameter expansions that require runtime evaluation. The
+# balance are always safe to trace.
+#
+# _AT_CHECK expands COMMANDS, but the Autoconf language does not provide a way
+# to safely expand arbitrary COMMANDS in an argument list, so the below tests
+# examine COMMANDS unexpanded.
+m4_define([_AT_DECIDE_TRACEABLE],
+[dnl Utility macros.
+m4_pushdef([at_reason])[]dnl
+m4_pushdef([at_lf], [
+])[]dnl
+dnl
+dnl Examine COMMANDS for a reason to never trace COMMANDS.
+m4_bmatch([$1],
+ [`.*`], [m4_pushdef([at_reason], [a `...` command substitution])],
+ [\$(], [m4_pushdef([at_reason], [a $(...) command substitution])],
+ [\${], [m4_pushdef([at_reason], [a ${...} parameter expansion])],
+ at_lf, [m4_pushdef([at_reason], [an embedded newline])])[]dnl
+dnl
+m4_ifval(m4_defn([at_reason]),
+[echo 'Not enabling shell tracing (command contains ]m4_defn([at_reason])[)'],
+[m4_bmatch([$1], [\$],
+dnl COMMANDS may contain parameter expansions; expand them an runtime.
+[case "AS_ESCAPE([$1], [`"\])" in
+ *'
+'*) echo 'Not enabling shell tracing (command contains an embedded newline)' ;;
+ *) at_trace_this=yes ;;
+ esac],
+dnl We know at build time that tracing COMMANDS is always safe.
+[at_trace_this=yes])])[]dnl
+m4_popdef([at_lf])[]dnl
+m4_popdef([at_reason])])
+
# _AT_CHECK(COMMANDS, [STATUS = 0], STDOUT, STDERR,
# [RUN-IF-FAIL], [RUN-IF-PASS], SHELL_ESCAPE_IO)
@@ -1228,20 +1332,7 @@ echo AT_LINE >"$at_check_line_file"
at_trace_this=
if test -n "$at_traceon"; then
- at_lf='
-'
- at_cmd_expanded="AS_ESCAPE_FOR_EXPAND([$1])"
- case "$at_cmd_expanded" in
- *\$\(*\)*) at_reason='a $(...) command substitution' ;;
- *\`*\`*) at_reason='a `...` command substitution' ;;
- *"$at_lf"*) at_reason='an embedded newline' ;;
- *) at_reason= ;;
- esac
- if test -n "$at_reason"; then
- echo "Not enabling shell tracing (command contains $at_reason)"
- else
- at_trace_this=yes
- fi
+ _AT_DECIDE_TRACEABLE([$1])
fi
if test -n "$at_trace_this"; then
diff --exclude Makefile.am -urp -X dontdiff ac-clean/lib/m4sugar/m4sh.m4
ac-simple_pexp/lib/m4sugar/m4sh.m4
--- ac-clean/lib/m4sugar/m4sh.m4 2004-12-13 14:46:03.000000000 -0500
+++ ac-simple_pexp/lib/m4sugar/m4sh.m4 2004-12-15 06:51:44.000000000 -0500
@@ -475,7 +475,7 @@ $as_unset $1 || test "${$1+set}" != set
# too.
-# AS_ESCAPE(STRING, [CHARS = $"'\])
+# AS_ESCAPE(STRING, [CHARS = $"`\])
# ---------------------------------
# Escape the CHARS in STRING.
m4_define([AS_ESCAPE],
@@ -483,15 +483,6 @@ m4_define([AS_ESCAPE],
m4_ifval([$2], [[\([$2]\)]], [[\([\"$`]\)]]),
[\\\1])])
-# AS_ESCAPE_FOR_EXPAND(STRING)
-# ----------------------------
-# Escape characters in STRING that have special meaning to the shell
-# within double quotes, but leave parameter expansions active.
-# These substitutions come from sed_double_backslash in GNU Libtool.
-m4_define([AS_ESCAPE_FOR_EXPAND],
-[m4_bpatsubsts([AS_ESCAPE([$1], [`"\])],
- [^\(\(\\\\\\\\\)*\\\\\)\$], [\1\\$],
- [\([^\\]\(\\\\\\\\\)*\\\\\)\$], [\1\\$])])
# _AS_QUOTE_IFELSE(STRING, IF-MODERN-QUOTATION, IF-OLD-QUOTATION)
# ---------------------------------------------------------------
diff --exclude Makefile.am -urp -X dontdiff ac-clean/tests/autotest.at
ac-simple_pexp/tests/autotest.at
--- ac-clean/tests/autotest.at 2004-12-14 22:30:02.000000000 -0500
+++ ac-simple_pexp/tests/autotest.at 2004-12-18 02:22:10.745290401 -0500
@@ -19,10 +19,13 @@ AT_BANNER([Autotest.])
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA
# 02111-1307, USA.
-# AT_CHECK_AT(TITLE, SUITE-CODE, [XFAIL-CONDITION])
-# -------------------------------------------------
+# AT_CHECK_AT(TITLE, SUITE-CODE, [XFAIL-CONDITION], [STATUS = 0],
+# [STDOUT := ignore], STDERR)
+# ---------------------------------------------------------------
# Create a new test named TITLE that runs a minimal Autotest test suite,
-# SUITE-CODE. If given, XFAIL-CONDITION is passed to AT_XFAIL_IF.
+# SUITE-CODE. Call AT_XFAIL_IF with XFAIL-CONDITION. STATUS and STDERR pass
+# directly to the AT_CHECK that call the minimal test suite. STDOUT is not
+# used, but it is reserved for future use.
m4_define([AT_CHECK_AT],
[
AT_SETUP([$1])
@@ -43,15 +46,15 @@ AT_DATA([mysuite.at], [$2])
# log file it generates will overwrite the log that the Autoconf test
# suite produces for this test case.
AT_CHECK_AUTOM4TE([--language=autotest -o micro-suite mysuite.at])
-AT_CHECK([./micro-suite], 0, [ignore], [])
-AT_CHECK([./micro-suite -v -x], 0, [ignore], [])
+AT_CHECK([./micro-suite], m4_default([$4], 0), [ignore], [$6])
+AT_CHECK([./micro-suite -v -x], m4_default([$4], 0), [ignore], [$6])
AT_CLEANUP
])
-# AT_CHECK_AT_TEST(TITLE, SUITE-SNIPPET, [XFAIL-CONDITION])
-# ---------------------------------------------------------
-# Like AT_CHECK_AT, but surrounds SUITE-SNIPPET with a boilerplate
-# AT_INIT, AT_SETUP, and AT_CLEANUP.
+# AT_CHECK_AT_TEST(TITLE, SUITE-SNIPPET, ...)
+# -----------------------------------------------------------------------
+# Wrapper for AT_CHECK_AT that surrounds SUITE-SNIPPET with a boilerplate
+# AT_INIT, AT_SETUP, and AT_CLEANUP and passes other arguments verbatim.
m4_define([AT_CHECK_AT_TEST],
[AT_CHECK_AT([$1],
[[
@@ -59,14 +62,14 @@ AT_INIT([artificial test suite])
AT_SETUP([my only test])
$2
AT_CLEANUP
-]], $3)])
+]], m4_shiftn(2, $@))])
# Here documents for these tests contain forbidden macros.
m4_pattern_allow([^AT_])
-## ------------------------------------------------------ ##
-## Newlines and command substitutions in test commandds. ##
-## ------------------------------------------------------ ##
+## ----------------------------------------------------- ##
+## Newlines and command substitutions in test commands. ##
+## ----------------------------------------------------- ##
AT_CHECK_AT_TEST([Truth],
[AT_CHECK([:], 0, [], [])])
@@ -111,6 +114,43 @@ bar
bar
], [])])
+
+## ------------------------- ##
+## ${...} in test commands. ##
+## ------------------------- ##
+
+# If this invalid parameter expansion capsizes the test suite, the entire
+# AT_SETUP ... AT_CLEANUP subshell will exit, and the commands it runs will
+# appear to have succeeded. Therefore, we verify a failing test case.
+
+AT_CHECK_AT_TEST([Invalid brace-enclosed parameter expansion],
+ [AT_CHECK([echo '${=invalid}'], 0, [wrong])], [false], 1, ignore, ignore)
+
+
+## ---------------------------- ##
+## M4 macros in test commands. ##
+## ---------------------------- ##
+
+# The last paragaph in the comment above _AT_DECIDE_TRACEABLE illustrates why
+# this test fails.
+AT_CHECK_AT_TEST([Multiline command from M4 expansion],
+ [m4_define([GNU], ['foo
+bar'])
+ AT_CHECK([echo GNU], 0, [foo
+bar
+], [])], [:])
+
+AT_CHECK_AT_TEST([Double-M4-quoted command],
+ [m4_define([GNU], ['foo
+bar'])
+ AT_CHECK([[echo GNU]], 0, [[GNU
+]], [])])
+
+
+## -------------------------------------- ##
+## Backslash-<newline> in test commands. ##
+## -------------------------------------- ##
+
AT_CHECK_AT_TEST([BS-newline in command],
[AT_CHECK([echo Auto"\
"conf], 0, [Autoconf