bug#38398: non-obvious SCM_EOF_VAL rationale

[Zefram]

The part of the Guile manual on the representation of immediate objects
says:

# -- Macro: SCM SCM_EOF_VAL
#     The Scheme end-of-file value.  It has no standard written
#     representation, for obvious reasons.

I disagree with the manual: the reasons for the EOF value having no
s-expression representation are not at all obvious.  It's fairly obvious
that it's a value that can't be returned by read-char, and therefore is
not itself a character, but that's quite a different matter.  The lack of
s-expression representation actually comes from the entirely unobvious,
and undocumented in Guile, use of the EOF value with the read function.

In the RnRS series, the concept of an EOF object appears in R2RS, and
remains essentially unchanged from there.  (The only difference is that
R6RS specifies that there is one EOF object, whereas all others allow
for multiple EOF objects.)  They all specify that if the read function
encounters EOF then it will return an EOF object, and in order to support
that usage they also specify that EOF objects can never be returned
by read.  This poor design precludes RnRS specifying read syntax for any
EOF object.  The relationship here is fairly obvious, but only once one
is aware of this rather surprising use of EOF objects by read.

The situation in Guile is more muddied.  Because Guile supports the
"#." syntax for read-time evaluation, it actually *is* possible for the
read function to return an EOF object without having reached EOF:

$ echo '#.(eof-object)' | guile-2.2 -c '(fluid-set! read-eval? #t) (use-modules 
(rnrs io simple)) (write (read)) (newline)'
#<eof>

This is technically a violation of RnRS, but I have no complaint
about breaking such an onerous rule in these circumstances where it's
necessitated only by such a poor design decision.  Anyway, it means
that the RnRS rationale for having no s-expression representation for
the EOF object *doesn't apply* to Guile.  There's also precedent, in
"#nil", for Guile extending read syntax beyond RnRS for immediate objects.
So it seems to me that you are quite free to invent some readable syntax
such as "#eof" for the EOF object.

So, to resolve this, firstly you should add to the documentation of
the read function some text about its behaviour on EOF (on which it
is currently silent).  Perhaps also add some text about the ambiguity
of read returning the EOF object.  Then you should remove the ", for
obvious reasons" part of the SCM_EOF_VAL documentation.  After that you
have a choice.  You could leave the lack of s-expression representation
unexplained.  Alternatively you could attempt an actual explanation, which
in the minimal form would be "so that without the use of the non-standard
read-time-evaluation facility it can't be returned by the read function
in non-end-of-file situations, which would cause an ambiguity".  For Guile
2.4 you could instead add a read syntax for it and document that.

-zefram