bug#11287: Various issues with the test suite framework

[Stefano Lattarini]

Reference: <http://debbugs.gnu.org/11287>

To sum up the actions related to this bug report:

On 04/20/2012 01:13 PM, Akim Demaille wrote:
> Hi!
> 
> I have seen that check-html will be removed, or rather moved
> into the contrib part, but there are a few issues:
> 
> - the target is not declared recursive, so one has to write
>  the bouncing target herself.
>
This isn't a big deal anymore, now that the user can define extra
recursive targets through the macro AM_EXTRA_RECURSIVE_TARGETS
(features introduced in commit v1.12.1-163-g3da1457); the test
'contrib/t/parallel-tests-html-recursive.sh' (introduced by commit
v1.12.1-191-g53d4984) should prove this assertion.

> - because of that, the "naive" implementation of check-html
>  that just bounces the right directories will miss the
>  dependencies that check features (in other words, make
>  check properly recursively builds before, while check-html
>  would just go into the typical tests/ directory, and then
>  bounce to "check", so it completely missed the recursion
>  in the other directories and their dependencies).
>
Adding 'check-html' to AM_EXTRA_RECURSIVE_TARGETS should take
care of this issue as well.

> - RECURSIVE_TARGETS cannot be highjacked by the users, that's
>  sad.  Could a $(USER_RECURSIVE_TARGETS) be added to
>  RECURSIVE_TARGETS so that we can add our own targets that
>  traverse the whole package?  Actually maybe this should
>  be an Automake option to add recursive targets, and
>  then TARGET-am and TARGET-recursive would be generated and
>  bound to the proper TARGET.
>
Addressed by commit v1.12.1-163-g3da1457 (with a different design
 but comparable functionalities); see this thread for more info:
<http://lists.gnu.org/archive/html/automake-patches/2012-06/msg00195.html>

> - the .log.html recipe features $$RST2HTML, but in '', so it is
>  useless.  And anyway $(RST2HTML) already does the job.
>
Agreed.  Fixed by commit v1.12.1-192-g18dac90.

> - "$(AM_RST2HTMLFLAGS) $(RST2HTMLFLAGS)" is missing.
>
Fixed by commit v1.12.1-193-g67abae4.

> - the test compiler does not appear as a dependency for
>  the test logs that use it.
>
Reporting what I've stated in an earlier reply:

    This makes sense, since the test compiler is not required to be
    an in-project script, and not even to be specified as an absolute
    path -- it can be a command like "sh" or "/usr/bin/env perl".

> (This is most useful when RECHECK_LOG =.)

> - it would be equally useful to be able to add dependencies
>  to a category of test drivers.  Say, with the documented
>  example:
> 
>     TESTS = foo.pl bar.py baz
>     TEST_EXTENSIONS = .pl .py
>     PL_LOG_COMPILER = $(PERL)
>     AM_PL_LOG_FLAGS = -w
>     PY_LOG_COMPILER = $(PYTHON)
>     AM_PY_LOG_FLAGS = -v
>     LOG_COMPILER = ./wrapper-script
>     AM_LOG_FLAGS = -d
> 
>  be able to add PL_LOG_DEPENDENCIES, PY_LOG_DEPENDENCIESS and
>  LOG_DEPENDENCIES.  I don't think EXTRA dependencies would
>  be useful.
>
This would be tricky to implement in the current Automake code base, and
IMHO for a too little benefit; so I'm going to label this part of the
report as a "wontfix".  However, it was easy to implement in Automake-NG:
<http://lists.gnu.org/archive/html/automake-ng/2012-05/msg00022.html>

-*-*-*-

All that considered, I'm closing this bug report now.

Regards,
  Stefano