bug#75812: 29.2.50; Support native-compiling package-install from batch Emacs

[Spencer Baugh]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Spencer Baugh <sbaugh@janestreet.com>
>> Cc: app-emacs-dev@janestreet.com,  acorallo@gnu.org,
>>    stefankangas@gmail.com,  75812@debbugs.gnu.org,  shipmints@gmail.com
>> Date: Tue, 29 Apr 2025 14:05:38 -0400
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> >> > Or why not install a native-comp-async-all-done-hook that will notice
>> >> > when compilations are done and allow the batch job to exit?  All you
>> >> > need is a means to wait until all the async compilations are finished,
>> >> > no?
>> >> 
>> >> How do you propose to do that?  I guess I could call kill-emacs from the
>> >> hook, but I have more code I want to run after the native compilation is
>> >> finished, and I can't move it all into the hook.
>> >
>> > Then don't call kill-emacs from the hook; instead, use 'throw', or run
>> > the rest of the code from the hook directly.
>> 
>> So a function like this?
>> 
>> (defun native-comp-async-wait-for-all-done ()
>>   (unless (zerop (comp--async-runnings))
>>     (catch 'native-comp-done
>>       (let ((native-comp-async-all-done-hook (list (lambda () (throw 
>> 'native-comp-done)))))
>>         (while (sit-for 999))))))
>> 
>> Could we add that?
>
> Why do we need to add it?  Can't you just use it in your code?

Since others in this thread have said they would like to be able to wait
monitor the completion of native compilation.  It seems like it would be
good to have a high quality implementation be part of Emacs.  (and
frankly, I don't think throwing out of a hook as a means of control flow
is a good idea, and I'd like to do something better than that)

>> Yes, it's a matter of compiling each package separately in parallel,
>> which is annoying to do if they all write files to a single directory as
>> a side-effect.
>> 
>> But, I think what I could do is compile each package with a different
>> directory prepended to native-comp-eln-load-path, and then once all the
>> packages are compiled, merge those directories into one omnibus
>> directory, which is bundled with my installed Emacs.
>
> I still don't understand what you are trying to achieve, but the above
> should work.

Great, thank you for confirming!

>> Would it be reasonable to merge those directories into the last entry of
>> native-comp-eln-load-path?  (e.g. /usr/lib/emacs/30.1.50/native-lisp)
>
> No!  The last directory is in many cases not writable by users, and it
> includes preloaded files, see the doc string of the variable.  The
> last directory should be left alone.  Instead, add another directory
> to the default value, between the user's eln-cache and the last
> member, and have your local packages' *.eln files there.

I'm doing this package precompilation at Emacs build time, and the last
directory is writable at that time.  After installation, of course, it
is not writable, but that doesn't matter since only these precompiled
package .elns are going to be written there.

In general, I have site-lisp that is byte-compiled and native-compiled,
and it seems like the right place to write .eln files for site-lisp is
the last directory on native-comp-eln-load-path.  After all, site-lisp
is effectively bundled with Emacs, set up at install time just like core
Elisp.  Is there some issue with doing that?