bug#46256: [feature/native-comp] AOT eln files ignored if run from build tree

[Andrea Corallo]

Pip Cet <pipcet@gmail.com> writes:

> On Fri, Mar 5, 2021 at 9:33 AM Andrea Corallo via Bug reports for GNU
> Emacs, the Swiss army knife of text editors <bug-gnu-emacs@gnu.org>
> wrote:
>> Eli Zaretskii <eliz@gnu.org> writes:
>> > So currently the only way to fill up a newly created subdirectory of
>> > native-lisp/ is to manually delete the *.elc files of all the files in
>> > lisp.mk's $shortlisp list, is that sufficient?
>>
>> Yes I think so.
>>
>> The trouble of using make for building such a system is that make is not
>> aware of the .eln filename, so it should be necessary to ask the Emacs
>> binary about that to create dynamically the precise (multiple target)
>> rule.  Not very practical IMO...
>
> I do wonder whether the whole filename scheme is really the best option.
>
> IIUC, and that's a big if in this case, the main motivation for using
> hashes in the .eln filenames is that dlopen() is broken and may return
> the same handle for subsequent dlopen()s of the same name, even if the
> underlying file changed in between.

Unfortunately this was only an unfortunate discover along the road...
this design predates that.

> Merely verifying that the ABI is correct could be done at runtime, so
> that's no reason to keep a hash in the filename.
>
> So my vague idea is this:
>
> 1. implement fixed_dlopen(), which keeps track of filenames that have
> been opened and, if necessary, creates a temporary file and loads that
> instead of its argument.
> 2. compile lisp/emacs-lisp/bytecomp.el to lisp/emacs-lisp/bytecomp.elc
> and native-lisp/emacs-lisp/bytecomp.eln

So it was at the beginning, I think we moved away from that before the
odd dlopen behavior.

> 3. add extra code in the top level function of each .eln to check that
> the ABI is correct.
>
> This would allow us to use standard make rules. It would also make
> .eln filenames predictable. It might even draw someone's attention to
> the fact that dlopen() is broken and make them fix it.
>
> I'm probably missing other good reasons for the hashed filename scheme.

Yep, this was discussed in length on emacs-devel, IIRC mainly on a long
standing thread called "native compilation the bird-eye view" (or
something close).

  Andrea