bug#66732: tree-sitter fontification doesn't update multi-line syntax reliably

[Dmitry Gutov]

On 13/12/2023 05:28, Yuan Fu wrote:
> > > c-ts-mode--emacs-set-ranges is registered as a range rule, so many 
> > > tree-sitter function calls it before doing anything to make sure 
> > > range is up-to-date. treesit-font-lock-fontify-region calls 
> > > treesit-update-ranges at the beginning of its body, and 
> > > treesit-update-ranges calls c-ts-mode--emacs-set-ranges.
> >
> > That seems to mean that any feature accessing the parse tree should 
> > call treesit-update-ranges first. Including 
> > syntax-propertize-functions and *-extend-region-functions, which we 
> > currently don't do.
> >
> > But which boundaries is it supposed to use? Should 
> > treesit--syntax-extend-region call treesit-update-ranges, when wait 
> > for the parser updates, then possibly call treesit-update-ranges again 
> > on the extended boundaries, and so on, until the parser stops sending 
> > notifications? Will it stop?
>
> It'll stop. If the ranges don't change, no reparse will happen. And only 
> buffer content change can cause range change. Reparse itself can't. So 
> at most you'll see reparse -> update ranges -> reparse.
>
> > Anyway, could you try my patch? Like I said, I'm not sure if the 
> > insufficient fontification I'm observing in c-ts-mode is due to the 
> > problem with the solution, or due to the other redisplay-related 
> > problems on my system.
>
> Yeah, it doesn't solve the problem in c-ts-mode regarding block comments.
>
> We might need to run (progn (force-parse) (update-ranges) (force-parse)) 
> before jit-lock-fontify-now and sytax-ppss.

Well... I've replaced

  (treesit-buffer-root-node (treesit-language-at (point)))

in treesit--syntax-extend-region with

  (treesit-buffer-root-node (treesit-language-at (point)))
  (treesit-update-ranges beg end)
  (treesit-buffer-root-node (treesit-language-at (point)))

and even tried commenting out the call to 'treesit-update-ranges' inside 
treesit-font-lock-fontify-region, but the result looks unchanged. And 
the region does get extended, according to my print-debugging inside 
font-lock-default-fontify-region.

Could you check if you're seeing the same?