Re: A read-based grep-like for symbols (el-search?) (was Do shorthands break basic tooling (tags, grep, etc)? (was Re: Shorthands have landed on master))

[João Távora]

João Távora <joaotavora@gmail.com> writes:

> João Távora <joaotavora@gmail.com> writes:
>
>>> (benchmark 1 '(dolist (dir load-path)
>>>                  (when (file-exists-p dir)
>>>                    (let ((files (directory-files dir t "\\.el\\'")))
>>>                      (dolist (file files)
>>>                        (unless (file-directory-p file)
>>>                          (with-temp-buffer
>>>                            (insert-file file)
>>>                            (read-from-string (buffer-string)))))))))
>>>
>>> Reports 2.5 seconds here.
>
>> Are these Emacs's files?
>
> Nevermind, just saw it's load-path.  Which makes sense.
>
> So I have to ask: is this an 'Emacs -Q' load-path?

Anyway, the good news is that:

 * avoiding read-from-string, and using read isntead
 * using a single buffer
 * using insert-file-contents (insert-file is for interactive use)
 * using the Emacs -Q load-path

speeds your snippet down from 2 to 0.2 seconds.

The bad news is that your snippet was only reading the first form in a
file, so when you read the whole file it takes 2.8 seconds on my corei7
machine to read the whole Emacs sources.

So performance wise, it's not bad but can probably be improved.  I don't think
it's gonna be stupidly hard because:

* this is one of those embarassingly parallel problems.  We could just spawn
  spawn `invocation-name' to use SMP

* designing a caching strategy seems fairly easy here.

João