Re: Is add-to-list supposed to work when lexical-binding is t?

[Kelly Dean]

Stefan Monnier wrote:
>> Section 11.9.3 (Lexical Binding) in the manual says "functions like
>> `symbol-value', `boundp', and `set' only retrieve or modify
>> a variable's dynamic binding".  Why?
>
>Because a variable is not the same thing as a symbol.
>
>For dynamic binding, you can somewhat blur the difference and use the
>symbol's value cell as "the content of the variable of that name"
>because let-binding just temporarily changes the only global value.
>
>With lexical scoping, a given variable name can have many different
>values at the same time so this is not an option.

I read the last bit as, "with lexical scoping, a given symbol (symbolic name) 
can be interpreted as many different variables in different scopes (and each 
variable can have multiple instances) ...", to synchronize terminology. And I 
call the things that variables are bound to "instances", not "bindings" like 
the docs do.

In the scope of (let ((x 'a)) ...) with lexical binding enabled, since the 
symbol x is interpreted as a lexical variable rather than as the global 
variable (the latter bound to the global instance, the value cell for the 
symbol x), I propose a quote-lex special form, where (quote-lex x), with e.g. 
"&x" or maybe "^x" as readable syntax, returns a reference to the current 
instance (current in time and recursion level) of the lexical variable, like 
"&x" does in C, instead of returning the symbol x (which, without context, is a 
reference to the global instance). Also change the set function to accept not 
only symbols, but also lexical instance references, and change symbol-value to 
either return the value of a symbol's value cell as usual if given the symbol, 
or return the value of a lexical instance if given a reference to the latter.

Then you could use argument-mutating functions, including the standard 
add-to-list function, without having to convert them into macros, and do e.g.:
(let ((x '(a))) (add-to-list &x 'b) x) -> (b a)

The pair of closures produced by wrap-lexical in my previous message (but the 
first closure should more simply be just the plain value; oops) imitates what I 
propose; a special form would be needed to actually return a reference to the 
lexical instance itself. My set-passed-lexical and get-passed-lexical imitate 
what I propose for set and symbol-value.

Furthermore, instead of having a separate quote-lex, maybe overload quote, and 
return the symbol as usual where it's used as the global variable, or return a 
reference to the current lexical instance where the symbol is used as a lexical 
variable. Then even current code that uses argument-mutating functions could be 
used unmodified, instead of having to convert their relevant uses of quote to 
quote-lex. And lexical binding won't cause the accidents I described in my 
previous message.

If overloading quote is a bad idea, then quote-lex should return a reference to 
the current instance of the given variable regardless of whether it's given a 
lexical or global variable; that means in the latter case, it does the same 
thing as quote, which means quote-lex can be used instead of quote in all 
places, except where a symbol needs to be returned despite that symbol also 
serving as a lexical variable in the same scope, because the symbol will be 
used as the global variable or as something other than a variable.

>> (let ((x '(a))) (add-to-list 'x 'b) x) -> (b a)
>
>Yup, this is asking for trouble.  Use `push' or `cl-pushnew' instead.

So, converting every argument-mutating function into a macro actually _is_ the 
right thing to do? That seems unnecessarily complicated, just a way of working 
around the lack of lexical quoting.