bug#70368: [PATCH] Use a dedicated type to represent interpreted-function values

[Stefan Monnier]

>> There are two kinds of incompatibilities it introduces, in my experience:
>> 
>> - "soft" incompatibilities for code which tries to display the kind of
>>   the object it receives (such as in a completion table that wants to
>>   add an icon indicating if something is a macro, a compiled function,
>>   etc...).  Such code will still work but may display less informative
>>   info because it may fail to recognize the new objects as being
>>   interpreted functions.
>> - "real" incompatibilities for code which digs inside the
>>   entrails of functions to try and extract specific information.
>>   This may fail when faced with the new interpreted functions
>>   but should be easy to fix.
>>   As long as such code only tries to extract info and does it via
>>   `help-function-arglist`, `documentation`, and `interactive-form`,
>>   there's no problem, but some packages may use code inherited from
>>   a long time ago when `help-function-arglist` didn't exist, or written
>>   by coders who didn't know better.  I know Hyperbole used to do that,
>>   but I believe that's been fixed since.
>> - "hard" incompatibilities for code which really digs inside the
>>   code of functions.  `vc.el` did that a long time ago,
>>   `kmacro.el` did as well until OClosures, and Buttercup (a NonGNU ELPA
>>   package) did until a few months ago.  There are probably one or two
>>   packages out there that will be affected like Buttercup would have
>>   been.  FWIW, the Buttercup case was a mix of "hard" and "soft": it
>>   really looked inside the code, but used that only to provide more
>>   informative messages and had a fallback case when the code was
>>   compiled, so it would still work OK.
>
> Some of the above should be in NEWS, I think, in the "incompatible
> Lisp changes" section.

I don't really know what more I could put in there.  The entry I added
states the actual changes and the above "two" are just
direct consequences.

Obviously I could copy&paste that text literally, but it seems to me
it'd just make etc/NEWS longer for no concrete benefit.  Or maybe I'm
just misunderstanding.  How do you imagine a user or developer is going
to make use of the above info?

>> Where would that be?  I don't see where we document the
>> #f(compiled-function ...) used for byte-code, which was my inspiration
>> for the #f(lambda ...).
>
> We have the "Printed Representation" node, and then the representation
> of each type is documented where the type is documented, in
> subsections of "Programming Types".  So somewhere there, I think,
> probably in "Function Type"?  And you probably want to review
> "Byte-Code Type" as well.

AFAICT we currently don't document the cl-print output in there.
I did (in the mean time) update the "Byte-Code Type" to clarify that the
printed representation is the same #[...] as byte codes, and I also
mention the #f(...) notation.

>> This should be an extremely rare occurrence, and `display-call-tree`
>> itself is not documented, so I doubt it's worth the trouble.
>> If you prefer I can keep it as `<function>`.
> I thing <function> is preferable, since <function-like list> is
> somewhat mysterious, and what you say above means explaining the fine
> differences will be largely a wasted effort.

OK.

>> >> +DEFUN ("closurep", Fclosurep, Sclosurep,
>> >> +       1, 1, 0,
>> >> +       doc: /* Return t if OBJECT is a function object.  */)
>> >
>> > If the doc string is correct, then why is the function called
>> > 'closurep'?  It's against mnemonic memory.
>> 
>> The term "closure" is kind of funny, indeed: often it's used to say
>> "this is a piece of code packaged with its environment", but in most
>> cases where it's used all functions are like that, so the precise
>> meaning of "closure" can be argued to be exactly the same as "function",
>> just with a different connotation.  Sometimes the connotation is to
>> insist on the fact that it captured some variables, but object it's used
>> to distinguish between the abstract notion of functions and their
>> runtime representation, where "closure" means "an object which
>> represents a function".
>> 
>> IOW, the docstring above is meant to say "a function *object*" rather
>> than "a *function* object".  Compare with `functionp`:
>> 
>>     Return t if OBJECT is a function.
>> 
>> Maybe I should say something like:
>> 
>>     Return t if OBJECT is a closure, i.e. a function object.
>> 
>> ?
>
> Yes, I think that'd be better.

In the end, I went with:

       doc: /* Return t if OBJECT is a function of type `closure'.  */)

and improved the docstring of the `closure` type, so we get a more
direct access to a short description of the various fields of byte-code
and interpreted-function objects.

> The problem is that "function value" can be interpreted as "value
> returned by a function".

Ah, right.  Thanks.

>> Because `make-byte-code` is the function that we already have which
>> creates PVEC_CLOSURE (previous called PVEC_COMPILED) objects.  It used
>> to be used exclusively to create byte-code functions but now that same
>> representation is used for interpreted function.  I could rename it but
>> we already have `make-closure` for something slightly different, and
>> I don't see much benefit to the rename.
>> I'll add a comment explaining why we use `Fmake_byte_code`,
>
> I think this also calls for augmenting the documentation of
> make-byte-code to the effect that it could now create closures as
> well.

I'm not very happy documenting it, because I think it's an
implementation accident resulting from the historical evolution of
the code.  IOW something we may want to fix.
People who want to create an interpreted function should use
`make-interpreted-function` rather than `make-byte-code`.


        Stefan