[Koha-devel] Test::Debugger

[Nathan Gray]

On Tue, Sep 24, 2002 at 01:46:41AM -0400, Pat Eyler wrote:
> Ding! Ding! Ding! We have a winner!  This is one thing that I'd really
> like to see more of in Koha.  If you were interested in writing tests
> (even just starting tests for the Koha modules), we'd be a lot better off.
> It would also help you learn more about the way Koha works.

It looks like Pat would would like me to start work on some test suites
for Koha.  Is the C4 directory the place to start?

Also, I have just applied for the namespace Test::Debugger on CPAN.  I
am interested in using this module for testing, because it creates a log
file on test failures, and is compatible with Test.pm (which comes
bundled with Perl).

I have not uploaded the module to CPAN because I want to make sure the
documentation makes sense.  As many of you could benefit from using a
test module, or might like to know how I am going to structure my tests,
I am including the documentation in this message.

If anyone would like more information about testing Perl modules in
general, or more information about Test::Debugger, please let me know
(preferably through this list).

For simplicity, I plan on using the subroutine interface to
Test::Debugger.

kolibrie

----------------------
Documentation follows:
----------------------

NAME
    Test::Debugger - Create Test Output and Generate Log Files

SYNOPSIS
      use Test::Debugger;
      my $t = Test::Debugger->new(
          tests    => 1,
          log_file => 'test.log',
      );

      # set the order of the parameters passed to 'ok'
      $t->param_order('ok' => [qw(self expected actual message error)]);

      $t->ok($expected, $actual, $description, $error_message);
      $t->todo($expected, $actual, $description, $error_message);
      $t->skip($because, $expected, $actual, $description, $error);

      $t->ok_ne($expected, $actual, $description, $error_message);
      $t->ok_gt($expected, $actual, $description, $error_message);
      $t->ok_ge($expected, $actual, $description, $error_message);
      $t->ok_lt($expected, $actual, $description, $error_message);
      $t->ok_le($expected, $actual, $description, $error_message);
      $t->ok_re($expected, $actual, $description, $error_message);

      # OR (more compatible with Test.pm)

      use Test::Debugger;
      plan(tests => 1, log_file => 'test.log');
      ok($actual, $expected, $message, $error);

DESCRIPTION
    Have you ever tried to debug a test script that is failing tests?
    Without too many modifications, your test script can generate a log file
    with information about each failed test.

    You can take your existing test script, and with (hopefully) very little
    effort, convert it to use Test::Debugger. Then re-run your modified test
    and view the log file it creates.

  Object-Oriented Interface

    Test::Debugger can be run using exported subroutines, or using an
    object-oriented interface.

    The object-oriented interface allows for easier access to some types of
    functionality, such as comparision operators other than 'eq'.

    You start by creating a test object.

      my $t = Test::Debugger->new(%params);

    The `new' method calls `plan' automatically, so you can pass your `plan'
    parameters directly to `new'.

    The three most basic methods are `ok', `todo' and `skip'. You can
    rearrange the order of the parameters to these methods by running
    `param_order'. You must run `param_order' for the testing subroutines to
    work as methods.

      $t->param_order('ok' => ['self', 'actual', 'expected']);

    The key here is the word *self*. The default param order does not
    include *self* as a valid parameter. The example above sets the valid
    parameters to pass to `ok', including the test object, the value we are
    testing, and the value we expect it to match against.

    A parameter order need only be assigned for `ok'. The `todo' and `skip'
    methods will mimic the order assigned for `ok', if they are undefined.

    Now we can set up our tests:

      $t->ok($actual, $expected);

    An explanation of each method can be found further down in this
    document.

  Subroutine Interface

    The interface for Test::Debugger is based loosely on the interface for
    Test.pm. You have four subroutines exported into your namespace: `plan',
    `ok', `skip', and `todo'.

    You should `plan', but it is not as strictly enforced as in Test.pm.

    The default order of arguments to `ok' and `todo' is:

      ok($actual, $expected, $message, $error, $operator);

    You must supply at least an `$actual' value, and you should supply the
    first three parameters. You can change the order of the parameters with
    `param_order'.

  Log File

    I generally name my test log file 'test.log', and add an entry to my
    Makefile.PL to remove it on 'make clean'.

      WriteMakefile(
          ...
          'clean' => {
              'FILES' => 'test.log',
          },
      );

    If a test fails, is skipped, or is marked todo (and fails), an entry is
    made to the log file. Each entry looks similar to this:

      t/todo.t 1 TODO.
      should fail as TODO
      ### Expected ###
      1
      ### Actual Results ###
      0

    The first part is the name of the test file. The number is the subtest
    within the test file. The word `TODO' means that this test was not
    expected to succeed, because code it relies on is not finished. This
    word could also have been `FAILED' or `SKIPPED'.

    On the second line, we see the `$message' provided to the test. We would
    see the `$error' on the third line, if that had been provided.

    Then the expected value and the actual value are displayed. An extra
    newline is appended to the actual value, in the log file, so that there
    is a blank line between entries.

  Extra Debugging

    If the test log file is not sufficient to debug your code, or your test,
    you may enable the `note' subroutine of Devel::Messenger and write to
    another file, perhaps called something like 'debug.txt'.

      local *Test::Debugger::note = note { output => 'debug.txt' };

    Test::Debugger will `note' the test it is working on. You can then place
    notes in your module, and they will be grouped by test number.

  Methods/Subroutines

    Most methods may be called as a subroutine, rather than as a method.

    new Creates a new Test::Debugger instance and returns a handle to that
        instance. Accepts a HASH of parameters, which is passed to the
        `plan' method.

        Not available as a subroutine.

    next_test_number
        Takes a number to store internally as the next test number.

          $t->next_test_number(1);

          $next_test = $t->next_test_number();

        Returns the number of the next test to be run.

    param_order
        Allows you to specify in which order you shall submit parameters.

          $t->param_order($method => address@hidden);

        The key words allowed in the address@hidden' array are:

          self      Test::Debugger object
          expected  the value you expect to get
          actual    the value you are testing
          message   description of the test
          error     an error message to log
          operator  Perl comparison operator

        You may include as many or few of these items as parameters, as long
        as `actual' is always present.

    plan
        Sets up some internal variables, such as where to log errors.

          $t->plan(
              'skip'             => $because,
              'skip_message'     => $explanation,
              'tests'            => $total_number_of_tests,
              'todo'             => address@hidden,
              'log_file'         => $filename,
              'start'            => $number,
              'param_order'      => {
                  'ok' => ['actual', 'expected', 'message', 'error'],
              },
          );

        If you do not include a `tests' parameter, you must print the line:

          "1..$number\n"

        Otherwise Test::Harness will not know how many tests there are.

        If the `$because' of the `skip' parameter is true, the entire test
        file will be skipped.

    ok  Takes parameters which may include a Test::Debugger object handle,
        an expected value, actual value, description of the test, error
        message, and the Perl comparision operator to use.

        *See `param_order' for information on specifying the order of
        parameters*.

        Compares the actual and expected values (or determines the truth if
        only the actual value is specified in `param_order'). Prints "ok
        $number\n" on success, or "not ok $number $message\n" on failure.

          $t->ok('Test::Debugger',ref($t),'testing value of ref');

        An expected value of `qr/regex/' will try to match `$actual =~
        qr/regex/'.

          $t->ok(qr/help/, 'help me', 'testing regex');

        Returns true if the test does not fail.

    ok_<operator>
        Takes the same arguments as `ok'. Prints the same output as `ok'.
        Allows for the use of operators other than 'eq' to compare expected
        and actual values. Available operators are:

          ne  not equal
          gt  greater than
          ge  greater than or equal to
          lt  less than
          le  less than or equal to
          re  regualar expression

        Each operator is converted to the numeric equivalent if both
        arguments contain nothing other than numbers.

        Not available as subroutines.

    skip
        Skips a single test if a condition is met.

          $t->skip($because, $expected, $actual, $description);

        Prints "ok $number skip: $description\n", if `$because' is true.
        Returns true if the test succeeds, or if it is skipped.

    todo
        Skips a single test, or marks it TODO, because you are still working
        on the underlying code.

          $t->todo($expected, $actual, $description);

        Output depends on the version of Test::Harness you are using. Always
        returns true.

AUTHOR
    Nathan Gray - address@hidden

COPYRIGHT
    Test::Debugger is Copyright (c) 2002 Nathan Gray. All rights reserved.

    You may distribute under the terms of either the GNU General Public
    License, or the Perl Artistic License.

SEE ALSO
    perl(1), Test::Harness(3), Devel::Messenger(3).