bug#67455: (Record source position, etc., in doc strings, and use this in *Help* and backtraces.)

[Stefan Monnier]

> Pretending the problem doesn't exist won't solve it.  In the ;POS...
> structures for a lambda, there are two pointers - one to the definition
> of the lambda, the other to the point of use.

Fancy.  Could you give me an example where I see this in play?
[ To help me understand also what you mean by "definition of the
  lambda" and "point of use"?  ]

I looked around but all I could see where position info like

    [foo foo.el 41 nil]

which point to "the definition" of the function.

>> BTW, AFAIK the above is conceptually what the byte-compiler does (except
>> it performs a few more transformations between `macroexp--expand-all`
>> and `strip-all-symbol-positions`).
> It is a bad idea to conflate these two radically different uses of SWPs.

In what way are they radically different uses of SWPs?

>> Is it the case that `cl-defmethod` generates a function whose source
>> position (partly) points to `generic.el:403` if `cl-generic.el` was
>> interpreted but not if it compiled?
> No, the intention is that the source positions are independent of whether
> the code is compiled.

Good.  So why do the interpreted and compiled cases need to be
"radically different uses of SWPs"?

>> > (defmacro foo (lambda bar)
>> >   `(cons ,lambda ,bar))
>
>> > expands to
>
>> > (macro closure (t) (lambda bar) ";POS^^^A^A^A [foo *scratch* 158 nil]
>> > " (list 'cons lambda bar))
>
>> IIUC your reader will make the `lambda` formal argument into an SWP.
>> Where is that SWP stripped?
>
> In macroexp--expand-all in the "guard arm" near the end.

How?  `macroexp--expand-all` will not be passed this `lambda` because
it's not an *expression*.

`eval-buffer` of a buffer containing the above defmacro does:

1 -> (macroexp--expand-all (defalias 'foo (cons 'macro #'{foo} (lambda 
(#<symbol lambda at 46> bar) ";POS

 [foo foo.el 41 nil]\n" `(cons ,lambda 
,bar)))))
| 2 -> (macroexp--expand-all 'foo)
| 2 <- macroexp--expand-all: 'foo
| 2 -> (macroexp--expand-all (cons 'macro #'{foo} (lambda (#<symbol lambda at 
46> bar) ";POS

 [foo foo.el 41 nil]\n" `(cons ,lambda ,bar))))
| | 3 -> (macroexp--expand-all 'macro)
| | 3 <- macroexp--expand-all: 'macro
| | 3 -> (macroexp--expand-all #'{foo} (lambda (#<symbol lambda at 46> bar) 
";POS

 [foo foo.el 41 nil]\n" `(cons ,lambda ,bar)))
| | | 4 -> (macroexp--expand-all ";POS

 [foo foo.el 41 nil]\n")
| | | 4 <- macroexp--expand-all: ";POS

 [foo foo.el 41 nil]\n"
| | | 4 -> (macroexp--expand-all `(cons ,lambda ,bar))
| | | | 5 -> (macroexp--expand-all 'cons)
| | | | 5 <- macroexp--expand-all: 'cons
| | | | 5 -> (macroexp--expand-all lambda)
| | | | 5 <- macroexp--expand-all: lambda
| | | | 5 -> (macroexp--expand-all bar)
| | | | 5 <- macroexp--expand-all: bar
| | | 4 <- macroexp--expand-all: (list 'cons lambda bar)
| | 3 <- macroexp--expand-all: #'{foo} (lambda (#<symbol lambda at 46> bar) 
";POS

 [foo foo.el 41 nil]\n" (list 'cons lambda bar))
| 2 <- macroexp--expand-all: (cons 'macro #'{foo} (lambda (#<symbol lambda at 
46> bar) ";POS

 [foo foo.el 41 nil]\n" (list 'cons lambda bar)))
1 <- macroexp--expand-all: (defalias 'foo (cons 'macro #'{foo} (lambda 
(#<symbol lambda at 46> bar) ";POS

 [foo foo.el 41 nil]\n" (list 'cons lambda 
bar))))

So we see that indeed it returns code where the formal argument `lambda`
is (incorrectly) a SYMPOS.  Yet somehow the sympos is stripped after
macroexpansion somewhere since `(symbol-function 'foo)` shows the
resulting function doesn't have any symposes in it.

>> > so it is clear this case is getting handled OK.  I'm afraid I can't
>> > point out the exact place in the code at the moment where this is
>> > getting done.
>> I think it would be good to know, so as to be able to decide whether
>> it'll indeed always work right, or we just got lucky this time.
> See above.

Yes, please, see above 🙂

>> Could you explain what you think makes it intrinsically complex?
> The mass of detail that needs dealing with that Emacs has collected over
> the decades.  As a counter question, why do you think the exercise ought
> to be simple (assuming you do think this)?

Because you solved the hard part when you added the symposes for the compiler.


        Stefan