[Groff] mdocmx(7): reference extension for the mdoc semantic markup language

[Steffen Nurpmeso]

Hello Ingo and the list,

i'm sorry for being so late, having first talked about "More
referenceability for -mdoc would be an improvement!?" on
2014-09-11 [1].  So here i present a first working prototype for
mdocmx(7), as above.

  [1] http://thread.gmane.org/gmane.comp.printing.groff.general/12302

Without too much blabla: if you read a manual and see a reference
that you want to follow you currently have to take several steps
in order to do so with the usual Unix manual system; whereas our
brains have created nice shortcuts after doing so many thousands
of times, our software has not.  This is why sapiens sapiens is so
successful: mother nature buffers and corrects all our faults.
Or at least something along these lines.

Mdoc(7) is the best language for manual pages that we have, as it
structures the text semantically.  Well, not comparable to the
SGML predecessor from the 60s (but the 60s and 70s were or must
have been the better times anyway), but much better than the
man(7) macros which only do some weak structuring.  But even if
you convert a correct and laborious mdoc(7) document to HTML or
PDF you will not gain a table of contents, an index or even
references, the latter not even inside the document itself.

To overcome this status quo many projects manage their manuals in
a different language, say, a SGML package, and convert to manual
pages, HTML and PDF just as necessary.  Even those projects suffer
from the necessity to install the manual in multiple formats in
order to improve the situation however, and of course the
installed manual page is as static and unresponsive as it always
has been.

Exactly the latter is what mdocmx(7) tries to improve for the
mdoc(7) semantic markup language.  What is mdocmx(7)?

  1. A single new multiplexer macro for mdoc(7): ".Mx".
     Note that ".Mx -enable"d documents immediately and without
     any further effort (except preprocessing) support anchors
     and references for mdoc(7) builtin commands like .Sh and .Ss
     on the one hand and .Sx and .Xr on the other.

     Documented in the attached mdocmx.7.
     Patch for groff(1)'s v1.22.3 ([2]) mdoc(7) attached as part
     of groff-1_22_3.diff.
     Status:

     . Documents instrumented with this extension seem to get
       swallowed by all tested mdoc(7) environments without
       causing any problems.  Of course, if warnings are enabled
       or similar modes are chosen in troff(1) or mandoc(1)
       warnings will occur.  But these are not deemed to be
       default modes of operation.

     . ".Mx -toc" is not yet expanded from within the macros, but
       requires a preprocessor (as below) run.

     . It works well for the shipped manuals and the authors MUA
       manual, but has yet a problem with the example shown in
       mdocmx(7), for example; i.e., there one reads '.Mx Ic
       "final anchor"' and that anchor is actually visible,
       whereas it shouldn't (it is not a reference).
       Thus: not mature yet.   (I blame mdoc(7) for that.  But
       anyway don't blame me!)

      . Dew-fresh, comments need revisit etc.  Shouldn't be made
        public, but i'm in a hurry...

      . E.g., could include PDF and HTML support macros and
        automatically support more output devices than just
        grotty(1), and different to that without even touching any
        C(++) code.

      [2] http://ftp.gnu.org/gnu/groff/groff-1.22.3.tar.gz

  2. A preprocessor, written in portable sh(1) and awk(1):
     mdocmx(1).  It will walk the document and collect .Mx
     sequences in order to create the anchors.  It will not
     perform any work, acting rather like cat(1), unless the
     document is a mdocmx(7) document that has a ".Mx -enable" in
     the document prologue.  It will not perform any work and also
     rather act like cat(1) if the document was already
     preprocessed by mdocmx(1).  The latter property of mdocmx(7)
     allows a developer to locally preprocess the manual and
     distribute the result.

     Note this preprocessor is only necessary for single-pass
     troff(1) implementations.  Multi-pass troff(1) as well as
     specialized utilities like mandoc(1) don't require it, of
     course.  (In fact mandoc(1) should only need the less(1)
     patch as below, and be standalone beside that.)

     Attached as mdocmx.sh.
     Documented in the attached mdocmx.1.
     Status: mature, though possibly a bit dumb.

  3. A patch for the less(1) pager for understanding new,
     specially crafted ^H backslash sequences.  Different to the
     (rather) well known ^H formatting sequences these are
     completely hidden (remain uninterpreted for display).

     Once in the pager, the new control sequence (for now) ^X-^R
     (control-X, then control-R) will display a prompt "[anchor]:"
     where the number of an anchor can be entered.  Then the
     document is searched and upon success the anchor will be
     scrolled into view.  Shall the anchor refer to a .Xr, i.e.,
     an external reference, "Read external manual: !man 7 mdoc"
     (replace the 7 and the mdoc with the manual in question) is
     displayed and one may either accept via RETURN or drop via
     rather anything else (BS, ESC,..).  In the former case the
     current pager instance is temporarily replaced in order to
     read the referenced manual; upon return from this, "!back
     from external manual  (press RETURN)" is displayed.

     Patch for less(1) v458 (the last stable release [3]) attached
     as less-v458.diff.
     Status: mature, but documentation missing.

     . The fine pager lv(1) works properly (if $GROFF_NO_SGR is
       set, since it doesn't understand ISO 6429/ECMA-48).
       A patch for it is possibly added once mdocmx(7) is mature.

     . A tested p(1) pager is however completely broken.

     . The protocol used is as follows (commented in patch too):

       -  Local anchor: '{DIGITS}', each char doubled and ^H'd
          away.

       - .Xr anchor: '{!SECTION;MANUAL}', each char doubled and
         ^H'd away.
         In fact an .Xr anchor is a normal local anchor, which is
         searched and being scrolled to, directly followed by the
         shown .Xr anchor.  No whitespace in between.

         To be the least intrusive possible we simply use the
         normal searching and scrolling capabilities: because of
         the scrolling .Xr anchors are not shared for multiple
         section/name matches, but a running anchor number is
         used.

      [3] http://www.greenwoodsoftware.com/less/less-458.tar.gz

  4. troff(1) output device extensions.
     For now grotty(1) is supported.

     Patch for groff(1)'s v1.22.3 ([4]) grotty(1) attached as part
     of
     groff-1_22_3.diff.
     Status: mature, but documentation missing.

      [4] http://ftp.gnu.org/gnu/groff/groff-1.22.3.tar.gz

Fine.  I hope you like it.  I do.  Except that the mdoc(7) macros
are not yet really done and the manuals need to be updated.
But for mandoc(1) all that doesn't matter anyway, it is off the
hook and can support nice stuff in less(1) right away.  :-(. :)
Patches apply simply via "cd DIR && patch -E -p1 < PATCH".
Have fun,

--steffen

mdocmx.7
Description: Text document

groff-1_22_3.diff
Description: Text Data

mdocmx.sh
Description: Bourne shell script

mdocmx.1
Description: Text document

less-v458.diff
Description: Text Data