Re: Autotest usage outside the Autoconf

[Akim Demaille]

>>>>> "Alexey" == Alexey Mahotkin <address@hidden> writes:

Alexey> Was it ever supposed that Autotest be used as general-purpose
Alexey> testing framework?

Yes it is, and it is currently my main focus in the Autoconf
development.  Please, be sure to use *CVS* Autoconf to use Autotest.
The documentation is weak, very weak, but I'm currently writing some
documentation for O'Reilly.  People who are interested should ask to
me privately for pointers, and I will deliver it to them, provided you
give some feedback :)


Alexey> If yes, then does any other package besides Autoconf use the
Alexey> Autotest for itself?

CVS Bison, CVS GNU M4.


Alexey> - writing idempotent test-cases is too hard sometimes: I'd
Alexey> like that the framework would not "now retry the test-cases
Alexey> verbosely and show you where it fails".  I'd like the
Alexey> test-cases be run verbosely, but hidden, for the first time;

I have the same desire, but I'm not sure how to write this.  Autotest
is still changing.


Alexey> - it is sometimes inconvenient that AT_CHECK() executes tests
Alexey> from scratch, and in fresh shell.  I'd like sometimes to
Alexey> execute one test in the same current directory that the
Alexey> previous test left it;

There is no such problem: the test groups (AT_SETUP/AT_CLEANUP)
provide such mini environments.


Alexey> - it would be nice if the output sometimes be horizontal
Alexey> instead of the vertical; because CVS testsuite has lots of
Alexey> tiny tests which are too boring to write out as
Alexey> AT_SETUP/AT_CHECK/AT_CLEANUP each, but it would be great if
Alexey> it'd say something like:

Alexey>   1: basic.at:5 1-a0, 1-a1, 1-a2, 1b, 1c, 1d, ok

Frankly, I currently don't care about aesthetics, we will see later
when the framework is really satisfying.


Alexey> - will ever the Autoconf add everything that is needed to the
Alexey> source tree?  I had to copy m4sh.m4 and m4sugar.m4 manually,
Alexey> and Makefile.am contains explicit commands for creating the
Alexey> `testsuite' file.  Also, what is the status of atgeneral.m4
Alexey> and atspecific.m4?  Which of them am I supposed to edit?

This period is over now.  Look at CVS Autoconf, AC_CONFIG_TESTDIR and
tests/Makefile.am.


Alexey> - more specific question: how could I use shell variables in
Alexey> file contents in AT_DATA?  It quotes the dollar-signs with
Alexey> backslashes...  How to turn it off?

Use a plain heredoc.  What is your purpose?