Re: Generation of tags for the current project on the fly

[Dmitry Gutov]

On 1/14/18 7:21 PM, Eli Zaretskii wrote:

> > OK, so if the user says yes, we "temporarily visit" to auto-generated
> > tags table. Then the user saves a file and that table get invalidated
> > (or via some other mechanism), and we want to index it again. Ask again?
>
> No, I think asking once per project should be enough.

Until the end of the current Emacs session? And ask again after restart?

What about if the user switches to a different project and then back?

> I mean the first time the tags table is required might very well be at
> the beginning of working on a project, at which time the project
> source tree is not yet in the cache.

Yes, and? The user will need it to be indexed either way, right?

There's also another optimization opportunity: performing reindexing in 
an asynchronous fashion, in the background (maybe after a timeout, too), 
after any file is changed and saved. This one comes with its own 
tradeoffs, though.

> > For instance, could you try to see how long takes the generation of the
> > file list alone? And populating the buffer with it. But without passing
> > it to etags.
>
> What Lisp shall I use for that?

To measure the full time:

(benchmark 1 '(progn (etags--project-tags-cleanup) 
(etags--maybe-use-project-tags)))

To measure the time to generate the list of files only:

(benchmark 1 '(all-completions "" (project-file-completion-table 
(project-current) (list default-directory))))

> > > Invoking 'find' will always be faster, as it's optimized for
> > > traversing directory trees.
> >
> > 'git ls-files' will probably be faster still.
>
> Yes, but that only works in Git repositories.

We can probably optimize for that use case these days. Git or some other 
VCS is usually in place, especially in non-toy projects.

> > > No, TAGS is a standard target in GNU Makefile's.
> >
> > OK, good to know. Two questions, then:
> >
> > - Can we make it output the tags to stdout?
>
> Not likely.  But you could just visit the TAGS file(s), no?

Hmm, there are reasons not to do that in general, but if the way we 
generate the files is known to be "right", they mostly disappear (except 
for the implementation complexity: doing it this way and using temporary 
files in the other case will require more code).

How do we figure which files to visit? Do we just visit src/TAGS and 
expect the rest to be 'include'-d.

> > - Can we detect than a given Makefile has a proper TAGS target (that can
> > output to stdout)?
>
> Maybe CEDET has something, but if not, searching for ^TAGS: should be
> easy.
>
> > Not sure yet how to handle the TAGS files inclusions, though.
>
> "make TAGS" should handle it, as it does in Emacs.

So these questions have answers, good.

Here's another one: considering the reindexing costs are not always 
negligible and depend on the size of a project, will there be actual 
benefit to using the proposed scheme in GNU projects like Emacs, GCC and 
others (those are the ones that use 'make TAGS')? Or is there a subset 
of them, at least, which we expect to benefit?