Re: [Groff] Status of the portability work, and plans for the future

[Eric S. Raymond]

Gunnar Ritter <address@hidden>:
> "Eric S. Raymond" <address@hidden> wrote:
> 
> > Note that .br/.nl, .ti, .ta, and .in are *not* in the portable set.
> > These cannot be translated structurally by doclifter, and man-to-HTML
> > translators tend to ignore them or give useless results as well.
> > . . .
> > I noted previously that \w is *not* portable.
> 
> I think you are right about the facts here, but I would
> tend to give an adverse recommendation. Even when these
> features are not translated, they can be very useful in
> nroff visual markup as long as no structural information
> hides behind them.
> 
> As I said in an earlier post, whether a request is safe
> or not depends on the context in which is used. One can
> safely use almost all requests if their context is pure
> visual nroff markup which does not hurt when omitted.

Um. Gunnar, your feedback has been both intelligent and constructive
in the past, and I was hoping you would be among those to offer a detailed
critique of the work plan...but I'll be damned if I can make out what policy 
you are actually recommending here.

The premise that has been evolving as this discussion progressed is
that portable requests have to pass one of two filters: either (1)
they have to be handled well by all the third-party viewers we are
willing to consider relevant (which IMO looks more and more like the
C man2html and its derivatives), or (b) they have to have a structural
translation that doclifter can pick up and use.

Part of my job as the maintainer of doclifter is to make sure that if
these sets don't coincide, none of the shortfall is doclifter's fault.
That is, my job is to ensure that the "doclifter-safe" set is a superset
of the "viewer-portable" set.  So far, that's true -- in fact, doclifter
interprets a larger subset of [gtn]roff than anyone else outside that
lineage has tried to do yet.

I was originally focused completely on the question of what requests
are doclifter-safe.  You are the person who persuaded me that
viewer-portability is equally important, or more so.  I have taken up
that thought and pursued it with zeal, which is why the portable 
set in this draft is smaller than it was in earlier ones.

But now you say "One can safely use almost all requests if their context
is pure visual nroff markup which does not hurt when omitted."  You
reverse the thrust of your earlier argument, and you do it in a way
that makes no sense to me!  

Because, in general, we *cannot know* in advance what markup will "not hurt
when omitted".  We can have some visual/structural rules of equivalence, 
like "whitespace in filled  versions can stretch or shrink" (HTML browsers
rely on this one), but there is no way to know in advance whether or not
failure to render (say) a .br tag will change a human's structural reading
of the document.

So what criterion for "portable" are you actually groping for here?
It had better not be one that requires strong AI... :-)

> > Fortunately. these can almost always be replaced by uses of .nf/.fi,
> > .RS/.RE, and tbl markup (which doclifter handles).
> 
> Maybe you should simply handle pairing .in requests like
> .RS/.RE?

I already thought of that.  I would already be doing it, but the
observed usage pattern of .in is just a little too irregular to make
mechanical translation effective or safe.

> I can only repeat: Do not codify .SY and .OP in their
> current form. They are completely insufficient even for
> pure POSIX synopsis markup, so they can only lead to a
> weird mix of structural and visual information within
> the synopsis section.

I think your complaint is justified with regard to .OP but not with
regard to .SY.  It can be read as a purely structural "command synopsis
begins here" tag,

> Since .de is in the list of portable request, there
> is nothing to say against their use as local shorthand
> notation in the groff manual pages, of course.

Good, I'm glad you don't object to that.

> > 1)  Once we know what the portable set is, groff itself should issue
> > warnings when a man page uses a non-portable feature.
> 
> I repeat, it should not do that since there are uses
> of exotic requests which are completely harmless.

See above.  We want man markup to not break third-party viewers.  *You* 
pointed out the importance of that.
 
> > 3) .SY/.OP/.EX/.EE/.DS/.DE will also be needed in Heirloom troff.  
> > This one is pretty obviously Gunnar's baby.
> 
> I am willing to implement .EX/.EE/.DS/.DE but not .SY/.OP
> for the reasons described before.

I could live with that result.  I think you should reconsider your
objection to .SY, however.

> > I favor setting a floor that includes Latin-1.
> 
> I am rather strongly against that. groff is one of the
> few remaining programs that are stuck without UTF-8
> support. Most Linux distributions I know have turned to
> UTF-8 as the default encoding for German locales years
> ago. Latin-1 is already a legacy encoding as far as use
> on Unix terminals is concerned. We should thus not base
> our manual page recommendations on it.
> 
> I think we should recommend not to use anything except
> US-ASCII and traditional troff special characters (minus
> those that are found to be non-translatable) in English
> language manual pages. One does not really need Latin-1
> for these anyway.

I am quite willing to defer to your judgment on this matter,
especially if Werner confirms it.  Your choice would make the portable
set smaller, meaning less work for me :-).

> > 2) When, in the portable-subset description, can we say that .EX/.EE,
> > .SY/.OP, and .DS/.DE should be considered portable and no longer 
> > need local definitions?
> 
> When they are implemented in AIX, HP-UX, Solaris, and
> all other systems about which software maintainers care.
> We should not "consider" something portable when know
> that it actually is not.

But here I think you are setting too high a hurdle.  I will be very surprised 
if AIX or HP-UX are relevant to *anything* other than a handful of aging
legacy systems in two years' time.  I admit that Solaris might have a bit
more life in it, but Solaris is also going to be the one easiest to get
an enhancement patch into.
-- 
                <a href="http://www.catb.org/~esr/";>Eric S. Raymond</a>