Re: mandoc -man -Thtml bug: inconsistent vertical space before .TP

[Alejandro Colomar]

On Tue, Oct 24, 2023 at 02:13:34AM +0200, Ingo Schwarze wrote:
> Hi Alejandro,
> 
> Alejandro Colomar wrote on Mon, Oct 23, 2023 at 04:30:58PM +0200:
> 
> > This got me curious about TQ, since mandoc(1) considers it "very rarely
> > used, even in GNU pages".
> > 
> > Ingo, you may want to reword that, since TQ was being used in the Linux
> > man-pages project,
> 
> Done, thanks for the heads up.
> I append the resulting commit below.
> 
> > and yesterday I wrote a patch to use it even more:
> > <https://www.alejandro-colomar.es/src/alx/linux/man-pages/man-pages.git/commit/?id=0962e38105076fffc6dae174a9026917c6a3941d>
> 
> Strange, i pulled from
>   https://git.kernel.org/pub/scm/docs/man-pages/man-pages
> and don't see such changes there, so i'm just judging from
> code inspection right now, without looking at formatted versions.

I'm delaying pushes to <kernel.org>, so that I can prevent some
embarrasing mistakes.  In <www.alejandro-colomar.es/src/> you can find
the latest commits.  Here's a look at what I have:

$ git log --oneline --graph
* 5d63f142a (HEAD -> contrib, alx/contrib) ioctl_pagemap_scan.2: Add page for 
pagemap_scan IOCTL
* 6fdb1c030 (alx/main, main) man*/: ffix (Use '.TQ' where appropriate)
* 59804ef61 getent.1: ffix (Simplify indentation)
* ad3a0ad80 (korg/master, master) ioctl_userfaultfd.2: Add UFFD_FEATURE_WP_ASYNC
$ git remote -v
alx     
ssh://www.alejandro-colomar.es/srv/src/alx/linux/man-pages/man-pages.git (fetch)
alx     
ssh://www.alejandro-colomar.es/srv/src/alx/linux/man-pages/man-pages.git (push)
korg    git@gitolite.kernel.org:pub/scm/docs/man-pages/man-pages.git (fetch)
korg    git@gitolite.kernel.org:pub/scm/docs/man-pages/man-pages.git (push)

In 'contrib', I apply patches from others, and also do my own work when
I don't expect it to need a branch.  (And when I need a branch, after I
finish, I'll take the changes to 'contrib' as if I was an external
contributor.)  This branch is unstable, and will sometimes be rebased to
fix any issues in either the diff or the commit logs.

I fast-forward 'main' to 'contrib' when 'contrib' has not seen changes
in a day or so.  That's my personal main branch, and while I try to not
rebase it, since not many people depend on it, I sometimes do so, if I
catch some bigger problem before pushing to <kernel.org>.  If anyone
would ask me to not rebase, I'd not do it.

Then I delay pushes from that one to <kernel.org> for another day or so.

> 
> I think that is a really bad patch.
> 
>  1. It gratuitiously makes the description of almost every option
>     longer by a whole line, which is a significant waste of screen
>     real estate.  It's further aggravating that due to long options,
>     most Linux manual pages already have an extra line for the
>     options at the start of each paragraph.  Now you are
>     doubling that from one to two wasted lines per option
>     compared to the same functionality on BSD.  The situation
>     becomes even more dire because Linux already tends to have
>     many low-utility options compared to BSDs, so you keep driving
>     density of useful information down and verbosity and fluff up.

I considered that problem, and somewhat agree.  I thought of reordering
the options so that the short option goes second, so the paragraph
starts in the same line (probably).  I think it would keep the total
number of lines similar to with just TP and commas.

Except that there are cases with more than two tags, in which case it
would still add vertical space.

> 
>  2. Your argument that this helps searching is a red herring.

It is a thing for me.  I've suffered the searchability problems myself.
However, it's often in pages from other projects, so I'll keep suffering
it...

>     The weakness of man(7) in searching has nothing to do with
>     the *formatting* of list item heads, it is caused by the
>     lack of *semantic markup*.  There would be no problem creating
>     search anchors for multiple .Fl topics on the same output line
>     in mdoc(7).  The only reason i did not do it is because it
>     is irrelevant for us since we barely have any of those
>     POSIX-violating long options.

Do any of those help searching through a pager like less(1)?  Or are you
talking of typeset pages?  I find searchability of manual pages quite
acceptable, even if just using heuristics.  Here's a way to get an index
of a manual page:

        $ MANWIDTH=72 man ldd | grep -e '^[^ ]' -e '^   [^ ]'
        ldd(1)                  General Commands Manual                 ldd(1)
        NAME
        SYNOPSIS
        DESCRIPTION
           Security
        OPTIONS
        BUGS
        SEE ALSO
        Linux man‐pages (unreleased)    (date)                          ldd(1)

With a bit more heuristics, you can even include a list of options
accepted by a command in that index (or alone):

        $ MANWIDTH=72 man ldd | grep -e '^[^ ]' -e '^   [^ ]' -e '^       -'
        ldd(1)                  General Commands Manual                 ldd(1)
        NAME
        SYNOPSIS
        DESCRIPTION
           Security
        OPTIONS
               --version
               -v
               --verbose
               -u
               --unused
               -d
               --data-relocs
               -r
               --function-relocs
               --help Usage information.
        BUGS
        SEE ALSO
        Linux man‐pages (unreleased)    (date)                          ldd(1)

In less(1) I use similar heuristics to find what I want: '/    --unused'
will get me exactly where I expect.

> 
>  3. Seperating two synonymous .Fl entries onto different .TP/.TQ
>     lines weakens semantic expressiveness further.  Even though
>     mandoc already contains special guessing logic in the HTML
>     formatter to treat .TQ and following .TP as part of the same
>     list, other formatters and other output modes may be less smart.

I explicitly only support groff(1) and mandoc(1).  Other formatters will
already have problems with other features, very likely.

>     I mean, those are not even the same macros, and yet you hope
>     for them to be rcognized as entries in the same list?
> 
>  4. Even the best HTML markup that is so far feasible results
>     in *two* list entries, one with an empty body and and only the
>     second one with the corresponding content.  While that may or may
>     not look superficially right (depending on the CSS), it certainly
>     isn't semantically correct and is likely to cause accessibility
>     grief, for example for blind people.

Do you know anyone blind who reads manual pages?  I would be interested
in knowing their opinion.

>     So, you want HTML formatters (and formatters to other output
>     languages that support semantic markup) to combine .TQ and
>     subsequent .TP not only into the same list, but also into the same
>     list element - but only if the body of the .TQ is empty, i guess?
>     So you want different macros to behave identically in some
>     ways (.TQ and .TP part of the same list) but the same macro
>     fundamentally different depending on its content.  The same
>     macro sometimes gets its own element and sometimes needs to
>     fuse into a different macro.
>     So much fun for implementers of formatting modules!

It's not that bad.  While theoretically one could use TQ for other
things, we only use it for multi-tag paragraphs.  AFAIK, that's the only
extended use in the Linux man-pages project[1], and that's the use
suggested by groff_man[_style](7):

   Paragraphing macros
                                                       ... .   In  man
       pages and other technical literature, definition lists are fre‐
       quently  encountered;  these can be set as “tagged paragraphs”,
       which have one (.TP) or more (.TQ) leading tags followed  by  a
       paragraph  that  has  an  additional indentation.  ...

So a parser should be something like

(1)  I've seen a TP;
(2)  Expect either a TQ or a paragraph.
(3)  If I see a TQ, append it to the list of tags, and goto (2).
(4)  If I see a paragraph, close this entry with it.


[1]  I think we might be using it also as a compact list where there are
     no paragraphs, but I'm not sure.  I think we mostly use PD 0 for
     that (maybe exclusively).  Branden is working on a LS/LE pair to
     replace that.

> 
>  5. On top of all that, i have a hard time to think of any macro
>     that has a more wicked failure mode than .TQ in case the
>     formatter does not support it.  The output visually looks
>     perfectly fine, and the reader gets no hint that the *most
>     important* information is missing.

groff(1) and mandoc(1) support it.  That covers the entire list of
formatters I care about at the moment.

> 
> Actually, part of the reason why i initially added that
> additional warning about .TQ was that i felt uneasy about it:
> less portable than for example .EX, less important than for
> example .UR, no semantic benefit, purely presentational intent,
> ad hoc house of cards on the foundation of .TP, which is
> already shaky enough in its own right, and a terrible failure
> mode.  So at that time in the past, i was quite happy to get
> the impression that it was rarely used and stressed that in the
> documentation, hoping people would behave reasonably and only
> use it in exceptionally dire cases where really nothing else
> could possibly help.
> 
> Either way, that sentence in our manual page clearly is no longer
> true even without that particular patch, so it's gone now.

Yup; it was already widespread in the Linux Manual Pages.

Cheers,
Alex

> 
> > TQ seems to be a sibling of TP.  Not sure if this will affect this
> > -Thtml bug in some way; my experiments seem good, but they weren't
> > exhaustive.
> 
> Yes, i believe mandoc -T html is already able to more or less
> cope with typical use cases of .TQ.  It seems likely though that
> you can construct less typical use cases that render badly to HTML
> (or any other semantical markup language for that matter).  You can
> be virtually certain that HTML output from any non-mandoc formatter
> will be atrocious right now, and any future output to any semantical
> markup language will almost certainly be bad initially unless whoever
> implements that future output mode in whichever program closely looks
> at mandoc code first and is aware that man(7) formatting requires
> intelligent and complicated guessing and cannot be done in any way
> that is based on any context-independent grammar.
> 
> Yours,
>   Ingo
> 
> 
> Log Message:
> -----------
> Remove the statement that .TQ was "rarely used even in GNU manual pages".
> 
> It appears the Linux Manual Pages project started to quite aggressively
> use .TQ ever for cases where it not only provides no value, but makes
> formatting worse even when it formats as intended.  It's also a bad idea
> to use it that aggressively because .TQ has particularly catastrophic
> formatting behaviour on formatters (other than groff and mandoc) that do
> not support it: It essentially has the effect of omitting the topic of
> the discussion from the formatted version of the manual page, but in
> such a way that it does not become apparent to the reader that anything
> is missing.
> 
> But whether this is wise or stupid is their problem and none of our
> business.  Either way, we should not call a thing "rarely used"
> after that is no longer true.
> 
> Thanks to Alejandro Colomar <alx at kernel org org>
> for making me aware that the statement is no longer true.

Thanks.

> 
> Modified Files:
> --------------
>     mandoc:
>         man.7
> 
> Revision Data
> -------------
> Index: man.7
> ===================================================================
> RCS file: /home/cvs/mandoc/mandoc/man.7,v
> retrieving revision 1.149
> retrieving revision 1.150
> diff -Lman.7 -Lman.7 -u -p -r1.149 -r1.150
> --- man.7
> +++ man.7
> @@ -440,7 +440,7 @@ unspecified, the saved or default width 
>  Like
>  .Ic TP ,
>  except that no vertical spacing is inserted before the paragraph.
> -This is a GNU extension and rarely used even in GNU manual pages.
> +This is a GNU extension.
>  .It Ic UC
>  Sets the volume for the footer for compatibility with man pages from
>  .Bx

-- 
<https://www.alejandro-colomar.es/>

signature.asc
Description: PGP signature