Re: pic anomalies

[Ingo Schwarze]

Hi Larry,

Larry McVoy wrote on Sun, Dec 29, 2019 at 03:23:33PM -0800:

> On a different topic, is there any sort of test suite for pic
> and the other parts of roff?

I'm not aware of any tests for pic(1), anywhere.

An extremely small number of tests (14 tests grand total) is scattered
all around the groff source tree, for example:

 - contrib/gdiffmk/tests/gdiffmk_tests.sh
 - contrib/hdtbl/examples/test-hdtbl.sh.in
 - contrib/mom/examples/test-mom.sh.in
 - src/roff/groff/tests/
 - tmac/tests/

You can run that suite with "make check" from the top directory,
but it is not much use because it is very fragile.  For example,
right now, after "git pull" and building from source, nine out
of the fourteen tests fail for me on OpenBSD-current, so it's at
least about 65% broken.

The mandoc test suite provides more extensive and more robust
coverage of groff even though that's not its main purpose; its
purpose is to test mandoc.  But since it uses groff to generate
lots of files of expected reference output, you can also use it to
find regressions in groff.  I routinely do that when working on
groff patches.  However, you must run OpenBSD to do that: the
maintainer targets needed to run groff only work on OpenBSD.
I did the work of making the mandoc tests portable, but i didn't
write a portable version of the maintainer targets.

The sequence to test groff on OpenBSD is:

  cd /usr/src/regress/usr.bin/mandoc
  make obj-clean
  make groff-clean
  make groff         # runs nroff on 457 test files
                     # and tbl on 24 test files
  cvs diff -Nup      # to see whether groff output changed
  # the rest is not needed when only testing groff
  make obj
  make regress       # to compare mandoc output to groff output;
                     # runs about 1300 mandoc tests grand total

For the code of this test suite, see

  https://cvsweb.openbsd.org/src/regress/usr.bin/mandoc/

This suite includes tests for some low-level roff stuff (request
and macro argument handling, conditionals, escape sequences, scaling
units, user defined strings, .br, .cc, .ce, .char, .de, .ds, .ft,
.ig, .it, .ll, .na, .nr and number registers, .po, .ps, .return,
.rm, .rn, .shift, .sp, .ta, .ti, .tr, .while, complete testing of
character escape sequences in -T ascii and -T utf8 output modes,
very extensive testing of the groff_mdoc(7) macros and also substantial
testing of the groff_man(7) macros, and about two dozen tests for
tbl(1).

Admittedly, the focus is on what is needed for manual pages
and on ASCII output.  There are no tests whatsoever for PostScript
or PDF output.

Yours,
  Ingo