bug#68246: 30.0.50; Add non-TS mode as extra parent of TS modes

[João Távora]

On Fri, Jan 5, 2024 at 3:34 PM Eli Zaretskii <eliz@gnu.org> wrote:

> > I _have_ thought about it.  And I started from evidence that
> > not everything a major mode dedicated to a language supplies
> > is directly related to the parser implementation. Many things
> > are, but not all.  So reimplementing a full major mode just
> > for changing the praser backend might not make sense.
>
> Experience shows that eventually most if not all of that goes back to
> how the mode parses or analyzes the text (a.k.a. "source code").

That might be.  That's why I specifically wrote "parser implementation"
It _needn't_ (and quite fortunately often _doesn't_) go back to that.

> > > How can you find common
> > > grounds between these so different bases for implementation?
> >
> > Very easily, I think.  Stefan's patch is one such example.
> > What it is fixing?  Tools that want this common ground and haven't
> > found it, of course!
>
> What Stefan's patch fixes is the features that depend only on the
> mode's symbol, but don't call any mode-specific functions or examine
> its data structures.

Yes.  But it doesn't fix a minor mode that relies on a buffer-local
setting of one of this minor-modes variables, and this setting
happens in the mode body or in a hook, say flymake-diagnostic-functions,
but there are many.  Same for a minor mode relying on syntax-ppss.

> > Take inserting comments via comment-dwim.
>
> Even comment-dwim already shows a problem: it must determine whether
> point is inside a comment, and TS and non-TS modes do that radically
> differently.

Again, that's true for the implementation-wise.  But it needn't
be of the  interface.

> LSP only cares for the language, so of course it can benefit from
> Stefan's patch, because all that matters is the mode's symbol.

Stefan's patch becomes irrelevant (for LSP) if we switch to
eglot-server-program  (singular).  Either we will have two identical
settings of  eglot-server-program in foo-mode and foo-ts-mode, or
we will use a shared one.

Yasnippet (another package I re-wrote mostly) also relies on this
"database" approach. It shouldn't, but I didn't know better at
the time.  Stefan's patch is only needed for it because refactoring
it is work that noone wants to put in (at least I don't).

> > Or consulting documentation.
>
> Again, only the mode's symbol is important.

No.  Say some consult-documentation minor-mode relies on
some setting of 'documentation-function'?

> > Or anything we've built (including muscle memory) that lives on top
> > of syntactic abstractions like forward-sexp.
>
> Here you already bump into a problem, because most languages have no
> notion of "sexp", so making a TS mode do the same as a traditional
> mode is not easy at all.

Of course they do!!  How else would electric-pair-mode have worked
for virtually every language for more than 10 years, or C-M-u,
C-M-SPC, etc etc?   e-p-m doesn't have knowledge of past and
future modes, yet it works.  How?

Well the reason why e-p-m and these things work today for most ts
modes is because they are also _using_ the Lisp/C parser based on
syntax tables and syntax-propertize-function.

So, in essence, TS modes currently use two parsers wasting CPU doing
a fair amount of duplicate work.  I suppose this waste would only
stop once syntax-tables are nullified for those modes.  But we can't
many syntax-ppss clients (e-p-m, symbol-at-point) hanging.

IMO There's an elegant out.  When one puts a suitable
syntax-propertize-function that consults ts nodes, like someone did
for c++-ts-mode (still very minimal though) we take full
advantage of TS and even enable things that are very complicated to
do without TS.

For example, in non-TS c++-mode it's hard to have e-p-m pair '<' with
'>' but only in C++ template contexts.  But in c++ts-mode it's within
reach.  It does needs both an addition to c++-ts-mode's s-p-function
and a bugfix to elec-pair.el: I'm looking into that.

> > Basically any preference that the major-mode expresses regarding an
> > orthogonal facility (minor mode or not) should, in principle, be
> > shared.
>
> I invite you to compare CC mode with c-ts-mode, and see for yourself
> how the common grounds are very small.  It seems surprising at first
> sight, but once you look at the code, it is very clear.

And this is mainly because CC mode is, well, rather corpulent software,
let's put it like that.  This is why I wrote it makes sense to start
from scratch for this one.

But would some kind of c++-base-mode hurt in some way? Presuming Alan
allows it, of course.

> > At the very least, it seems a common hook would be useful, and that's
> > what an empty foo-base-mode() would give.
>
> Where a base mode makes sense, sure.  But even that causes problems,
> since the base mode leaves some stuff not set up.

I don't follow.  Can you give an example of a problem?  In fact
I'm happy to see exactly the strategy I suggested is already done in
ruby-mode.el and ruby-ts-mode.el.  What problems are caused by it?

It makes sense for the base mode be abstract of course: meaning
we should flag an error if calling 'foo-base-mode' detects it is called
outside of the the context of 'foo-concrete-mode'.

>  and this various
> things that you'd want to do in a mode hook are impossible in the
> base-mode hook.

I don't follow this part either.  Can you give an example using, say
the existing ruby-base-mode.

In summary, my position is that regardless of Stefan's patch, which
I'm not opposed to, we should:

1. Use add-derived-mode-parents sparingly and consider foo-base-mode when
possible.

2. have a remove-derived-mode-parent (for the other bug)

3. perhaps tighten up what we mean by derived-mode-p in the docs

João