bug#77745: Async native compilation runs every time I start Emacs

[Björn Lindqvist]

> > I tried doing exactly that. Here is the entirety of my early-init.el:
> >
> >     (startup-redirect-eln-cache (expand-file-name "~/.cache/emacs/eln"))
> >
> > warnings, icons, and cl-lib are still being recompiled every time
> > Emacs starts. So it doesn't matter how I modify the cache path, the
> > result is still redundant recompilation.
>
> Why does Emacs need to emit some warnings during startup in your case?
> That shouldn't normally happen.  If Emacs loads warnings.el, itr means
> it needed to emit some warning.

I don't know. In git Emacs I'm getting warnings like:

    Missing ‘lexical-binding’ cookie in "~/p/ConfigurationFiles/dot-emacs".

which I've never seen before. Emacs 30.1 does not emit lexical binding
cookie warnings, but the native compiler still recompiles the three
files I enumerated. Could it be that these three files are
gzip-compressed?

> > No, but the native compiler is running asynchronously and is delayed
> > in relation to the modules loaded at startup.
>
> I don't understand what you mean by "delayed".  The async native
> compilation is started as soon as some Lisp package is loaded which
> has no .eln file that Emacs can find.  That compilation is performed
> by a fresh Emacs session, which runs in batch mode and asynchronously
> wrt your interactive session.  But the value of
> native-comp-eln-load-path that the async compilation gets is the value
> which was known to Emacs at the moment it started the async
> compilation sub-process, and it doesn't change afterwards because
> whatever changes in user-emacs-directory and other variables that the
> interactive session does are not reflected to the compilation
> sub-process.  And since the compilation sub-process runs with -Q, it
> doesn't process your init files, so it uses the original unmodified
> values of user-emacs-directory and native-comp-eln-load-path.
>
> Does this explain what you see?

Yes and no. In particular, "it doesn't change afterwards because
whatever changes in user-emacs-directory and other variables that the
interactive session does, are not reflected to the compilation
sub-process", is, afaict, incorrect.

This is what I observe: Emacs starts and sees that there is no
warnings*.eln in either ~/.config/emacs/eln-cache/31.0.50/ or
$prefix/lib/emacs/31.0.50/native-lisp/ so it schedules warnings.el for
native compilation. Native compilation runs and then it writes the
warnings.eln file to ~/.cache/emacs/eln/31.0.50/.

So variables that the interactive session updates *must* be reflected
to the compilation subprocess. Otherwise it would have written
warnings.eln to the first entry of the original
native-comp-eln-load-path directory;
~/.config/emacs/eln-cache/31.0.50/

> > My guess is that it schedules those modules for recompilation that
> > it cannot find in its load path but then, crucially, writes them to
> > the new path it won't look through the next time Emacs starts.
>
> How many execution units does your CPU have? what does the function
> num-processors return on that machine?

16

> > I cut off the rest of your message because, while your advice is
> > appreciated, it is not about solving the bug.
>
> What is the bug, in your opinion?

Well, I think you agree that the path Emacs uses for the eln cache
should be user configurable? Some users want it in ~/.cache/emacs
others are fine with it in ~/.config/emacs. You should be able to set
it without causing redundant recompilation. I can think of two ways of
fixing this bug:

1. Ensure that the native compiler writes to the same directory it
reads from. E.g., it looked for warnings.eln in
~/.config/emacs/eln-cache and it will write to
~/.config/emacs/eln-cache, regardless of concurrent updates to
native-comp-eln-load-path (given what you wrote this seems to have
been the intention).

2. Ensure that early-init.el is evaluated before the eln cache is
accessed. This seems cleanest to me, but idk if it is practical.


-- 
mvh/best regards Björn Lindqvist