Re: treesitter local parser: huge slowdown and memory usage in a long file

[Yuan Fu]

> On Feb 11, 2024, at 1:53 PM, Vincenzo Pupillo <v.pupillo@gmail.com> wrote:
> 
> Hi, 
> as a benchmark for my php-ts-mode (in 2 variants: one with tree-sitter-phpdoc 
> for php comment block, and another using regular expression for comment 
> block) 
> I use tcpdf.php (from the tcpdf library). This php file has 24730 lines and 
> generates 669 parser ranges, 665 of which are for phpdoc. 
> 
> As you can see from the profile (I try to edit the comment on line 2350) that 
> I 
> attached, the problem is in treesit--pre-redisplay.
> I tried playing around with the code a bit but to no avail (for example, I 
> limited treesit-update-ranges to window-start and window-end). 
> comments say:
>    ;; Force repase on _all_ the parsers might not be necessary, but
>    ;; this is probably the most robust way.
> 
> Any ideas?
> My php-ts-mode (It's a working progress) is available here: 
> https://github.com/vpxyz/php-ts-mode
> 
> Thanks
> V.
> 
> p.s without phpdoc emacs is as fast as with short php files.
> p.p.s. nvim with treesitter is as slow as my major mode with this file.
> 
> GNU Emacs 30.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.41, cairo 
> version 1.18.0) of 2024-02-11

cpu_profiler_report.txt
Description: Text document

mem_profiler_report.txt
Description: Text document

Thanks, the culprit is the call to treesit-update-ranges in 
treesit--pre-redisplay, where we don’t pass it any specific range, so it 
updates the range for the whole buffer. Eli, is there any way to get a rough 
estimate the range that redisplay is refreshing? Do you think something like 
this would work?

    (treesit-update-ranges
     (max (point-min) (- (window-start) 1000)) ; BEG
     (min (point-max) (+ (or (window-end) (+ (window-start) 4000)) 1000))) ; END

I guess the window-start would be outdated in pre-redisplay-function...

Yuan