bug#49836: Support ripgrep in semantic-symref-tool-grep

[Dmitry Gutov]

On 18.09.2021 17:18, Mattias Engdegård wrote:
> 18 sep. 2021 kl. 16.14 skrev Eli Zaretskii<eliz@gnu.org>:
>
> > > If the actual command (ripgrep or something else) takes zero seconds, what if 
> > > anything prevents a crisp snappy response from Emacs?
> > The sub-process communications infrastructure, of course.
> Yes, very likely, but also post-processing the output from the search process 
> (matching, sorting, etc) and preparing it for display. It would be interesting 
> to see a break-down and to see what if anything can be down to make to go 
> faster.

Before you dig in with OS-level debugger, here are things to try to 
narrow down the problems:

- Do the search for a common term (many matches).
- Do the search for a rare term (with 1-2 matches).

See how the inside/outside timings compare. I'm guessing the latter case 
should be snappy, with a relatively small ratio. Whatever overhead Emacs 
has, will probably be on the sub-process infrastructure.

To investigate the former case (many matches), I suggest starting with 
'benchmark-progn'.

Wrapping the 'process-file' call will measure the shell invocation and 
getting the output into the buffer.

diff --git a/lisp/cedet/semantic/symref/grep.el 
b/lisp/cedet/semantic/symref/grep.el
index 53745b429a..7db5e79c91 100644
--- a/lisp/cedet/semantic/symref/grep.el
+++ b/lisp/cedet/semantic/symref/grep.el
@@ -163,8 +163,9 @@ semantic-symref-perform-search
       (let ((cmd (semantic-symref-grep-use-template
                   (directory-file-name (file-local-name rootdir))
                   filepattern grepflags greppat)))
-        (process-file semantic-symref-grep-shell nil b nil
-                      shell-command-switch cmd)))
+        (benchmark-progn
+          (process-file semantic-symref-grep-shell nil b nil
+                        shell-command-switch cmd))))
     (setq ans (semantic-symref-parse-tool-output tool b))
     ;; Return the answer
     ans))

Measuring the subsequent semantic-symref-parse-tool-output call can also 
show something, but it's usually fast.

Wrapping the xref--convert-hits call alone inside 
xref-references-in-directory should be more interesting: its work is 
less trivial. I don't know how much further it can be optimized, but 
help is welcome, of course.

diff --git a/lisp/progmodes/xref.el b/lisp/progmodes/xref.el
index 69cabd0b5a..ab0476b2bb 100644
--- a/lisp/progmodes/xref.el
+++ b/lisp/progmodes/xref.el
@@ -1548,9 +1548,11 @@ xref-references-in-directory
          (inst (semantic-symref-instantiate :searchfor symbol
                                             :searchtype 'symbol
                                             :searchscope 'subdirs
-                                            :resulttype 'line-and-text)))
-    (xref--convert-hits (semantic-symref-perform-search inst)
-                        (format "\\_<%s\\_>" (regexp-quote symbol)))))
+                                            :resulttype 'line-and-text))
+         (search-hits (semantic-symref-perform-search inst)))
+    (benchmark-progn
+      (xref--convert-hits search-hits
+                          (format "\\_<%s\\_>" (regexp-quote symbol))))))

 (define-obsolete-function-alias
   'xref-collect-references