[gpsd-dev] The agony, the ecstacy, the NTP documentation

[Eric S. Raymond]

Harlan Stenn <address@hidden>:
> "Eric S. Raymond" writes:
> > On the other hand, we *must* explain fudge factors, and how and why
> > one sets them. This is something which AFAICT the existing documentation
> > does extremely badly, if at all.
> 
> I'm very happy to receive patches, and happy to receive pointers to
> where existing NTP documentation  sucks.  http://bugs.ntp.org

The thought that I should fix this has actually been bothering me for
some years.  Unusually for a software engineer, I'm a better document
author/editor than even A-list technical writers; with this power 
comes responsibility, and I budget a certain percentage of my time
to fix what I think are the most serious documentation problems
within my reach even if they're on other peoples' projects.

My weighting formula for serious is "criticality to the
infrastructure" divided by "comprehensibility of the document".  In
effect I try to weight by implied downstream risk if I don't do
something. By that metric, NTP's documentation is pretty high on my
list of Things That Need Fixing.

The NTP documentation is *terrible* in a way that is completely
unsurprising if one knows the history.  It has expanded in fits
and starts from aides-memoire written by Dave Mills, who was well
known for his tendency to write in an eccentric private jargon
leaving out assumptions perfectly clear to him but not necessarily
to anyone else.

As it was expanded, later authors (whoever they were) piled on detail
in a Millsian manner while giving almost no thought to motivating
introductions.  None of it ever seems to have been reorganized for
clarity or systematically edited.  The writing is remorselessly
mediocre-to-bad in a way all too typical of engineer brain dumps.

The resulting whole achieves a level of impenetrability that reminds
me of classical Chinese literature or the Big Dumb Objects in SF
puzzle stories. It requires brain-burning effort to understand even
partially, and getting inside its assumptions seems to render people
so incapable of remembering what it was initially like to confront the
enigma that they can't see where it needs to be clarified (Gary Miller
is by his own report Exhibit A for this phenomenon).

I've known I should try to fix this for years. The combination of
technical background, writing skills and sheer bloody-minded
persistence required is so unusual that there can't be more than three
or people alive who'd be really right for the job.  It is disturbingly
possible (though I fervently hope it isn't true) that the fit set
is a singleton consisting of me.

Nevertheless, I have not sailed in proposing to fix this for three reasons:

1) I've been pretty busy with other things.

2) The comprehension task is difficult even for me, a 30-year hacker
with strong chops in adjacent technical areas.  I've tried to get a
grip on it at least twice before and failed.  This tells me that
fully grokking it will be hard.  Not impossible; nothing like this
can stop me if I'm motivated enough. But *hard*.

3) Really fixing it can't be done with point patches - people behaving
on the assumption that point patches suffice are in fact a major
reason it's in such awful shape.  It needs a *rewrite* - it needs to be
restructured totally, with careful attention to distinctions between 
tutorial material, references, and practice documents.

The project is doable, but would require a minimum of two man-months 
of hard work, total support from the project owners, a hacker/writer as
capable as me, and experts on call to assist that writer.

I have not yet been able to justify trying to pull together the time
and resources for this.  To be honest, I still can't.  Other things
need doing even more.

But I've written this screed because *you* need to understand that as
long as you think in terms of "receiving patches", you will have failed
to understand the magnitude and nature of the problem.
-- 
                <a href="http://www.catb.org/~esr/";>Eric S. Raymond</a>