Re: advice needed for multi-threading patch

[Stefan Monnier]

> Yeah.  What I meant here is that specbind has a little extra code in it
> to install a Lisp_ThreadLocal object when needed.  So, if the C code
> uses specbind to do let-like binding, everything will work fine.  If the
> C code does not do this, then things will break.

I see, yes of course it's a problem.  And it's worse than that: the same
holds for *all* global variables, whether they are exported to Lisp
or not.

Stefan> I think it should, unless some of the buffer-local variables are
Stefan> let-bound by the thread.

> I've been thinking about this a bit.  It is trickier than it seems,
> because a thread can actually have let-bindings for buffer-locals from
> multiple buffers at the same time.  There's an example of this in the
> elisp manual.  (This is also an area the threading code does not handle
> well yet.)

> I think this means it does not suffice to keep a simple per-thread count
> of buffer-local bindings.

Indeed.  The lock should probably be on the buffer-local binding itself
(i.e. the combination of the buffer and the variable).

> While thinking about this I realized that (I think) the current code
> does the wrong thing if one thread let-binds a variable and then another
> thread calls make-variable-buffer-local on it.

Calling make-variable-buffer-local or make-local-variable when the
variable is let-bound is a serious problem, yes.  IIRC we already have
bugs in such cases in the current code.
AFAICT, the only good answer is to disallow such things (e.g. signal an
error in make-variable-buffer-local and make-local-variable when the var
is let-bound).
I already took a tiny step in that direction when I added a warning to
defvar when called while the variable is let-bound.

> It is hard for me to see how this could be done in a compatible way.

Agreed.

> Right now elisp operates with few constraints; an elisp program can call
> select-frame, which seems to imply that per-frame or per-keyboard
> threads can't be done.

The way I think of it is that select-frame would send a message to the
relevant frame, along with a function to run there, and the destination
to which to send the result when the function is done.

But how to link it with the dynamic and foo-local scoping of
let-bindings is still far from clear.

> One thing that would help me is having a concrete idea of what subset of
> features would make this work be actually useful.  I mostly implemented
> it because it was cool, and because Giuseppe's initial patch convinced
> me that it was possible (before that I'd written it off as totally
> impractical).  Now that a proof of concept works it would be nice to
> have a concrete goal.

Some goals could be:
- run Gnus in its own thread.  This should be easy-ish since Gnus
  shouldn't interact with pretty much anything else.
- split Gnus into several threads (e.g. in my case, the most important
  one would be a thread to do the "g", i.e. refresh the *Group* buffer,
  which with IMAP boxes can takes ages).
  This is a lot more difficult since this thread would need to interact
  with the main Gnus thread.
- move the jit-lock-context to a thread.  Might be non-trivial, but at
  least the code is small.
- move some VC operations to threads (e.g. vc-checkin).
- run some of the buffer-parsing code in threads (e.g. the one in
  CEDET's semantic, or the one in js2-mode).


        Stefan