bug#62368: 29.0.60; Evaluating predicates before creating captured nodes in treesit-query-capture

[Dmitry Gutov]

Hi Yuan!

On 22/03/2023 06:49, Yuan Fu wrote:
> X-Debbugs-CC:dgutov@yandex.ru
>
> Dmitry, when you have time, could you try your benchmark in bug#60953
> with this patch? I made predicates evaluate before we create any nodes,
> so #equal and #match should be more efficient now, when there are a lot
> of rejections. In the same time #pred is made slightly worst since they
> now create a lisp node and discard it. (But this can be fixed with a
> little more complexity.)

Thank you, I was curious what would the improvement be if we could delay 
allocation of node structures until :match is checked.

But for my benchmark the difference is on the order of 4-5%. It seems we 
are scraping the barrel in terms of improving allocations/reducing GC 
because according to 'benchmark-run', where the whole run of a 100 
iterations of the scenario takes ~1.1s, the time spent in GC is 0.150s. 
And the improved version takes like 1.04s, with 0.1s in GC.

So if you ask me, I think I'd prefer to hold off on applying this patch 
until we either find scenarios where the improvement is more 
significant, or we find and eliminate some other bigger bottleneck 
first, after which these 5% grow to become 10-20% or more, of remaining 
runtime. The current approach is pretty Lisp-y, so I kind of like it.

And there's the issue of #pred, of course, which which could swing the 
difference in the other direction (I didn't test any code which uses it).

We could also try a smaller change: where the initial list of conses for 
result is build with capture_id's in car's, and then substituted with 
capture_name if the predicates all match. Then tthe treesit_node 
pseudovectors would still be created eagerly, though.

Here's the current perf report for my benchmark, most of the time is 
spent in libtree-sitter:

  17.02%  emacs         libtree-sitter.so.0.0       [.] 
ts_tree_cursor_current_status              ◆
  10.94%  emacs         libtree-sitter.so.0.0       [.] 
ts_tree_cursor_goto_next_sibling           ▒
   9.93%  emacs         libtree-sitter.so.0.0       [.] 
ts_tree_cursor_goto_first_child            ▒
   9.55%  emacs         emacs                       [.] 
process_mark_stack                         ▒
   4.56%  emacs         libtree-sitter.so.0.0       [.] 
ts_node_start_point                        ▒
   3.90%  emacs         libtree-sitter.so.0.0       [.] 
ts_tree_cursor_parent_node                 ▒
   3.69%  emacs         emacs                       [.] 
re_match_2_internal                        ▒
   3.08%  emacs         libtree-sitter.so.0.0       [.] 
ts_language_symbol_metadata                ▒
   1.61%  emacs         emacs                       [.] exec_byte_code 
                           ▒
   1.47%  emacs         libtree-sitter.so.0.0       [.] 
ts_node_end_point                          ▒
   1.44%  emacs         libtree-sitter.so.0.0       [.] 
ts_tree_cursor_current_node                ▒
   1.13%  emacs         emacs                       [.] 
allocate_vectorlike                        ▒
   1.11%  emacs         emacs                       [.] sweep_strings 
                           ▒
   1.04%  emacs         libtree-sitter.so.0.0       [.] 
ts_node_end_byte                           ▒
   0.94%  emacs         emacs                       [.] next_interval 
                           ▒
   0.91%  emacs         libtree-sitter.so.0.0       [.] 
ts_tree_cursor_goto_parent                 ▒
   0.88%  emacs         emacs                       [.] 
lookup_char_property                       ▒
   0.81%  emacs         emacs                       [.] find_interval 
                           ▒
   0.68%  emacs         emacs                       [.] 
pdumper_marked_p_impl                      ▒
   0.67%  emacs         emacs                       [.] assq_no_quit 
                           ▒
   0.56%  emacs         libtree-sitter.so.0.0       [.] ts_node_symbol 
                           ▒
   0.56%  emacs         emacs                       [.] mark_char_table 
                           ▒
   0.55%  emacs         emacs                       [.] execute_charset 
                           ▒
   0.49%  emacs         libtree-sitter.so.0.0       [.] 
0x000000000001ae3e                         ▒
   0.49%  emacs         emacs                       [.] re_search_2 
                           ▒
   0.48%  emacs         emacs                       [.] funcall_subr 
                           ▒
   0.46%  emacs         libc.so.6                   [.] __strncmp_sse42 
                           ▒
   0.42%  emacs         libtree-sitter.so.0.0       [.] 
ts_language_public_symbol                  ▒
   0.41%  emacs         libtree-sitter.so.0.0       [.] 
ts_node_is_named                           ▒
   0.40%  emacs         libtree-sitter.so.0.0       [.] ts_node_new 
                           ▒
   0.34%  emacs         emacs                       [.] Fassq 
                           ▒
   0.34%  emacs         emacs                       [.] sweep_vectors