bug#77745: Async native compilation runs every time I start Emacs

[Björn Lindqvist]

Hello again,

I tried setting native-comp-eln-load-path explicitly in init.el

    (setq native-comp-eln-load-path
      (list (expand-file-name "~/tmp/")
            "/opt/emacs/lib/emacs/31.0.50/native-lisp/"))

and this works much better. The native compiler only
writes to ~/tmp/31.0.50-859de2ee/. Not in any directory under
~/.config/emacs. Only the files warnings-28e75f4d-f0ade81c.eln,
icons-eafe82eb-47a1ab50.eln, and cl-lib-8b938900-6d6e9142.eln gets
recompiled on startup.

> What this means is that if your init files load packages, directly or
> indirectly, before they modify user-emacs-directory, the native
> compilation will start for those packages using the value of
> native-comp-eln-load-path that is not what your session will
> subsequently use.  And packages compiled after user-emacs-directory is
> modified will have their *.eln files deposited in another directory.

> IOW, the order in which you modify user-emacs-directory and load
> packages that need native compilation matters _a_lot_, as you have
> discovered.

The first sexp of my init.el is

    (setq user-emacs-directory "~/.config/emacs/lisp/")

which is (?) executed *before* the native compiler kicks in. Unless
something odd happens during *parsing* of init.el, but that seems
far-fetched.

What I think happens is that the native compiler builds the
native-comp-eln-load-path based on the value of user-emacs-directory
before init.el is loaded. Then user-emacs-directory gets modified and
this changes the path it *writes* to, but does nothing to the paths it
*reads* from. So it compiles the files and writes them to a directory
it won't read from the next time I start Emacs. Indeed, if I log
native-comp-eln-load-path I can see that the value is:

    (~/.config/emacs/eln-cache/ /opt/emacs/lib/emacs/31.0.50/native-lisp/)

but the native compiler writes to ~/.config/emacs/lisp/eln-cache/
(note: /lisp/), contradicting the manual: "By default,
asynchronous native compilation writes the *.eln files it produces to
a subdirectory of the first writable directory specified by the
native-comp-eln-load-path variable".

So it must be a bug that setting user-emacs-directory updates the path
the native compiler writes to, but not the path(s) it reads from.

> We recommend not to modify the value of user-emacs-directory,
> because doing that might break various initializations Emacs
> performs during startup.

Are there technical reasons user-emacs-directory can't be changed?
Isn't code that relies on user-emacs-directory having a specific value
or to never change is buggy?

> The eln cache Emacs uses is not really a "cache" XDG spec had in mind.
> The *.eln files are not "non-essential", since we want them to
> persist, and removing that directory will make the next startup of
> Emacs significantly longer and more expensive.

I think what you're describing is exactly what a cache does and
exactly how other programs use $XDG_CACHE_HOME. Of course a program
would take longer to start after you blow up its cache.


--
mvh/best regards Björn Lindqvist