man page portability (was: [PATCH 4/9] man/clear.1: Migrate macro usage conventions.)

[G. Branden Robinson]

[Thomas: I spitball some ideas for streamlining ncurses man page
generation starting at "I may end up piloting" below.  I don't have
mastery of the ncurses build process, so if something's a non-starter,
I'd appreciate hearing about it.]

Hi Ingo,

At 2023-10-02T20:50:23+0200, Ingo Schwarze wrote:
> Yes, see https://man.openbsd.org/mandoc#DIAGNOSTICS

I should, uh, sit down and read mandoc(1) from top to bottom one of
these days.  I have to admit that the X/Open Curses Issue 7 document
(all 400+ pages of it) elbowed its way to the front of my queue
recently.

[...]
> Consequently, in autoconfiguration systems, *never* test for version
> numbers or program names, always use feature tests as specific as
> possible.

I agree with this.

> Needless to say, feature testing is not possible in manual page source
> code, but that doesn't change the fact that attempting to test for
> program names (with .g) or version numbers is more likely to cause
> frustration to all involved parties than to help with good formatting.

...but not (entirely) with this.  One could always format the page and
parse the output, and not necessarily as plain text--as troff(1) output.

Having done this in some groff unit and regression tests, I concede that
it is tedious, highly sensitive to irrelevant changes, and should not be
undertaken as a first resort.

> If you *really* want to support systems running old Kernighan troff,
> then you have no reasonable choice but to use 1979 Version 7 UNIX
> man(7) syntax and nothing else, which obviously implies that your
> manual pages will not benefit from any language and formatting
> improvements developed since then, and hence will be less useful
> on modern systes.  Yes, that's a tradeoff.  Trying to be extra clever
> and combining the best of both worlds in the manual page source code
> is more likely to actually result in combining the worst of both
> worlds - unreliability on both historical and modern systems.

Again, I don't completely agree.  The _entire_ *roff feature set was
generally understood to be available to all man pages everywhere until
sometime in the mid-1990s when people started trying to write man2html
translators.  Arguably, mandoc(1) author Kristaps Dzonsons was the most
destructive force to good man page typography that the world has yet
seen, because his approach decreed so much of the *roff language
verboten, and because his efforts enjoyed as much success as they did.
Perhaps that success should be credited largely to you, as you've been
maintaining mandoc for many years and compromised the purity of the
original vision severely, by doing things like handling string and macro
definitions, and conditional requests (`if`, `ie`, `el`) at all.

(And I am not saying that's a bad thing--it's a good one.)

On the other hand, I felt in 2017 and still feel today that keeping the
size of the language that a man page author needs to know small is
important to the goal of getting people to write them well (or at all).

On the third hand, mandoc(1) partisans tend also to be mdoc(7)
partisans, and that macro language is anything _but_ small.

> I see one way out that is likely to yield both good results on modern
> systems and passable results on historical systems.  Write your pages
> using a reasonably modern version of the language, such that they
> work well on modern systems and are easy to maintain.  Install the
> manual pages as-is by default, containing not a single .de and not
> a single .if.
> 
> When you use a modern feature - say, for example, .EX/.EE - and
> get reports that it doesn't work on their system from some of your
> users: write a configuration test for the feature, run that test
> at the ./configure stage, and if the configuration test fails, run
> a custom script that downgrades the feature in the installed pages
> during installation.  For example, your script might replace .EX with
> something like
> 
>   .nf
>   .ft CW
> 
> and .EE with something like
> 
>   .ft R
>   .fi
> 
> It doesn't need to be fancy, no need to bother supporting nesting
> of .EX in itself or inside surrounding .ft or .PD etc. etc. because
> it's only for your pages and you know how you use .EX in your pages.
> Avoiding unusual nesting in your pages is trivial - apart from the
> fact that it is also a bad idea in general.
> No need to publish such a script because i don't think any other
> project will need it: the requirement here feels so extremely unusual.

I may end up piloting some patches along these lines.  There are some
puzzling anomalies in the way ncurses generates its man pages.  They're
unrelated to the _syntactical_ man page content portability issue, so
this part may be of more interest to Thomas than to you.

1.  man/MKada_config.in and man/MKncu_config.in are processed at
    _configure_ time to generate file with names like
    "man/adacurses6-config.1" and "man/ncurses6-config.1", respectively.
    (The names are parameterized in the ABI revision number.)

    This is both good and bad.  It means that stuff that can be
    determined at configuration time gets determined and used.  On the
    other hand, make(1) dependency tracking is absent, so if you change
    the sources of these man pages (in the ".in" files), they don't get
    regenerated.

2.  Other stuff that is determined at configuration time _isn't_ handled
    this way.  For instance, the section number of the "file formats"
    portion of the manual is system-dependent.  (Or once was, or could
    be--I can't find the logic that actually rewrites it.  If that
    replacement is defunct, then the ncurses man pages could be
    profitably de-parameterized in this respect.

    Another example is the location of the terminfo database.

    It looks like the termcap database might have been intended for such
    parameterization as well, but either this was abandoned or it was
    just a copy-and-paste error from infotocap(1) to captotinfo(1).
    There is some evidence for the latter.

    https://lists.gnu.org/archive/html/bug-ncurses/2023-09/msg00073.html

    As far as I know, there was only ever one canonical location for the
    termcap database: /etc/termcap.  If you had another, you were
    expected to set the TERMCAP environment variable.

3.  A lot of other man page content is determined at configuration time
    but not applied to the man page files then, _or_ when "make" is run;
    the text of most ncurses man pages take their final form when the
    "install" target is run, whereupon in many configurations they are
    compressed as well.  I find that this makes my revision cycle for
    ncurses man page edits needlessly long.

Ideally--to my mind--the ncurses man pages wouldn't use any of the above
3 approaches.  Configuration determines all parameterized text in the
man pages.  These substitutions can all be written to a Makefile or sed
script.  The man pages would (mostly)[1] move to ".in" files, and suffix
rules could direct make(1) how to generate .1, .3, .5 and .7 files from
"{1,3,5,7}.in".

If this sounds familiar to anyone, that might be because it's how groff
handles a similar problem.[2]

(There is still an issue of altering the file name suffix to match the
needs of the target system--oddball stuff like "infocmp.1m" and
"wscrl.3ncurses"--_those_, I would handle at installation time, since
the installed files might be getting a ".gz" suffix anyway.)

> > and mandoc(1) _doesn't_ define it but nevertheless supports many
> > groff extensions.
> 
> That's not true, mandoc defines a .g read-only number register
> in exactly the same way as groff, see the test file below.

My bad.  I attempted to discover this by using `tm`, was annoyed by the
fact that mandoc(1) doesn't support `tm`, and immediately gave up.

Prematurely, yes--but you might consider supporting `tm` while I'm
heaping things onto your plate.  I can't be the _only_ person who'd like
to use it to inspect internal state, and it has to beat supporting the
other debugging requests (`pm`, `pev`, `pnr`, `ptr`).

(By the way, mandoc_roff(7) does not document `pnr` as a groff
extension, but it should.)

[snip]

At any rate, I have a much simpler proposed revision to clear(1).  It
still punches through the floor to `nf`, `fi`, and `ft` requests in a
minor way but (a) should achieve the desired formatting on AT&T
nroff/troff (except for C/A/T output with Unix Version 7 troff); and (b)
uses `EX`/`EE` macros where the implementation supports them
already, doing only harmlessly redundant things.

I'll post that separately.

Regards,
Branden

[1] Construction of terminfo(5) is a little more complex, but it occurs
    to me that it, too, could be made one source file, and a sed(1) "r"
    command could read in the terminal capability list at the
    appropriate place.

[2] https://git.savannah.gnu.org/cgit/groff.git/tree/Makefile.am?h=1.23.0#n831
    https://git.savannah.gnu.org/cgit/groff.git/tree/makevarescape.sed?h=1.23.0

signature.asc
Description: PGP signature