bug#46256: [feature/native-comp] AOT eln files ignored if run from build tree

[Andrea Corallo]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: Eli Zaretskii <eliz@gnu.org>, 46256@debbugs.gnu.org,
>>         andrewjmoreton@gmail.com
>> Date: Thu, 04 Mar 2021 14:49:47 +0000
>> 
>> I've a reproducer that is most luckily due to the same issue you are
>> observing:
>> 
>> emacs -batch -l comp -f batch-native-compile 
>> .../emacs/lisp/progmodes/cc-engine.el
>> 
>> GC kicks-in and we end-up marking #<subr c-string-list-p>, we try then
>> to mark its compilation unit but we segfault (backtrace below).
>
> AFAICT, the crash I see here, while compiling subr-x.el, is not inside
> GC: gc_in_progress is zero when Emacs crashes.
>
> To make debugging easier, I started Emacs like this:
>
>   emacs -batch -l comp -f batch-byte-native-compile-for-bootstrap 
> ../lisp/emacs-lisp/subr-x.el
>
> (AFAIU, using batch-byte-native-compile-for-bootstrap is currently the
> only way of invoking the native compilation in the same Emacs process,
> not in an async subprocess, is that right?)

Correct

> It crashes inside comp--compile-ctxt-to-file, and when it does, the C
> stack seems to be smashed:
>
>   Thread 1 received signal SIGSEGV, Segmentation fault.
>   0x06acac3e in ?? ()
>   (gdb) bt
>   #0  0x06acac3e in ?? ()
>   #1  0x00010101 in ?? ()
>   Backtrace stopped: previous frame inner to this frame (corrupt stack?)
>
>   Lisp Backtrace:
>   "comp--compile-ctxt-to-file" (0x82ca78)
>   "comp-compile-ctxt-to-file" (0x82cc88)
>   "comp-final1" (0x82cfb0)
>   "comp-final" (0x82d238)
>   "comp--native-compile" (0x82d468)
>   "batch-native-compile" (0x82d6a0)
>   "batch-byte-native-compile-for-bootstrap" (0x82d908)
>   "command-line-1" (0x82e360)
>   "command-line" (0x82ef08)
>   "normal-top-level" (0x82f630)
>
> I then put a breakpoint in comp--compile-ctxt-to-file and stepped
> through it.  This behaves erratically: if I just step with "next", it
> seems to crash inside the call to gcc_jit_context_set_int_option,
> here:
>
>   gcc_jit_context_set_int_option (comp.ctxt,
>                                 GCC_JIT_INT_OPTION_OPTIMIZATION_LEVEL,
>                                 comp.speed < 0 ? 0
>                                 : (comp.speed > 3 ? 3 : comp.speed));
>
> But if I "stepi" inside gcc_jit_int_option_optimization_level, it
> somehow seems to return, proceeds to this line:
>
>   gcc_jit_context_compile_to_file (comp.ctxt,
>                                  GCC_JIT_OUTPUT_KIND_DYNAMIC_LIBRARY,
>                                  SSDATA (tmp_file));
>
> Then something goes wrong inside it: the backtrace shows bogus
> addresses (note the "0xbaadf00d" thingies):
>
>   0x0766f180 in ?? ()
>   (gdb) bt
>   #0  0x0766f180 in ?? ()
>   #1  0x672f756e in ?? ()
>   #2  0x652f7469 in ?? ()
>   #3  0x7363616d in ?? ()
>   #4  0x74616e2f in ?? ()
>   #5  0x2d657669 in ?? ()
>   #6  0x706d6f63 in ?? ()
>   #7  0x74616e2f in ?? ()
>   #8  0x2d657669 in ?? ()
>   #9  0x7073696c in ?? ()
>   #10 0x2e38322f in ?? ()
>   #11 0x30352e30 in ?? ()
>   #12 0x3238312d in ?? ()
>   #13 0x65306335 in ?? ()
>   #14 0x75732f32 in ?? ()
>   #15 0x782d7262 in ?? ()
>   #16 0x6432302d in ?? ()
>   #17 0x33666566 in ?? ()
>   #18 0x37312d32 in ?? ()
>   #19 0x62656166 in ?? ()
>   #20 0x47736431 in ?? ()
>   #21 0x47305561 in ?? ()
>   #22 0x6e6c652e in ?? ()
>   #23 0x706d742e in ?? ()
>   #24 0xbaadf000 in ?? ()
>   #25 0xbaadf00d in ?? ()
>   #26 0xbaadf00d in ?? ()
>   #27 0xbaadf00d in ?? ()
>   #28 0xbaadf00d in ?? ()
>   #29 0xbaadf00d in ?? ()
>   #30 0xbaadf00d in ?? ()
>   #31 0xbaadf00d in ?? ()
>   #32 0xbaadf00d in ?? ()
>   #33 0xbaadf00d in ?? ()
>   #34 0xbaadf00d in ?? ()
>   #35 0xbaadf00d in ?? ()
>   #36 0xbaadf00d in ?? ()
>   #37 0xbaadf00d in ?? ()
>   #38 0xbaadf00d in ?? ()
>   #39 0xbaadf00d in ?? ()
>   #40 0xbaadf00d in ?? ()
>   #41 0xbaadf00d in ?? ()
>   #42 0xbaadf00d in ?? ()
>   #43 0xbaadf00d in ?? ()
>   #44 0xbaadf00d in ?? ()
>   #45 0xbaadf00d in ?? ()
>   #46 0xbaadf00d in ?? ()
>   #47 0xbaadf00d in ?? ()
>   #48 0xbaadf00d in ?? ()
>   #49 0xbaadf00d in ?? ()
>   #50 0xbaadf00d in ?? ()
>   #51 0xbaadf00d in ?? ()
>   #52 0xbaadf00d in ?? ()
>   #53 0xbaadf00d in ?? ()
>   #54 0xbaadf00d in ?? ()
>   #55 0xbaadf00d in ?? ()
>   #56 0xbaadf00d in ?? ()
>   #57 0xbaadf00d in ?? ()
>   #58 0xbaadf00d in ?? ()
>   #59 0xbaadf00d in ?? ()
>   #60 0xbaadf00d in ?? ()
>   #61 0xbaadf00d in ?? ()
>   #62 0xbaadf00d in ?? ()
>   #63 0xbaadf00d in ?? ()
>   #64 0xbaadf00d in ?? ()
>   #65 0xbaadf00d in ?? ()
>   #66 0xbaadf00d in ?? ()
>   #67 0xbaadf00d in ?? ()
>   #68 0xbaadf00d in ?? ()
>   #69 0xbaadf00d in ?? ()
>   #70 0xbaadf00d in ?? ()
>   #71 0xbaadf00d in ?? ()
>   #72 0xbaadf00d in ?? ()
>   #73 0xbaadf00d in ?? ()
>   #74 0xbaadf00d in ?? ()
>   #75 0xbaadf00d in ?? ()
>   #76 0xbaadf00d in ?? ()
>   #77 0xbaadf00d in ?? ()
>   #78 0xbaadf00d in ?? ()
>   #79 0xbaadf00d in ?? ()
>   #80 0xbaadf00d in ?? ()
>   #81 0xbaadf00d in ?? ()
>   #82 0xbaadf00d in ?? ()
>   #83 0xbaadf00d in ?? ()
>   #84 0xbaadf00d in ?? ()
>   #85 0xbaadf00d in ?? ()
>   #86 0xbaadf00d in ?? ()
>   #87 0xbaadf00d in ?? ()
>   #88 0xbaadf00d in ?? ()
>   #89 0xbaadf00d in ?? ()
>   #90 0xbaadf00d in ?? ()
>   #91 0xbaadf00d in ?? ()
>   #92 0xbaadf00d in ?? ()
>   #93 0xbaadf00d in ?? ()
>   #94 0xbaadf00d in ?? ()
>   #95 0xbaadf00d in ?? ()
>   #96 0xbaadf00d in ?? ()
>   #97 0xbaadf00d in ?? ()
>   #98 0xbaadf00d in ?? ()
>   #99 0xbaadf00d in ?? ()
>   #100 0xbaadf00d in ?? ()
>   #101 0xbaadf00d in ?? ()
>   #102 0xbaadf00d in ?? ()
>   #103 0xbaadf00d in ?? ()
>   #104 0xbaadf00d in ?? ()
>   #105 0xbaadf00d in ?? ()
>   #106 0xbaadf00d in ?? ()
>   #107 0xbaadf00d in ?? ()
>   #108 0xbaadf00d in ?? ()
>   #109 0xbaadf00d in ?? ()
>   #110 0xbaadf00d in ?? ()
>   #111 0xbaadf00d in ?? ()
>   #112 0xbaadf00d in ?? ()
>   #113 0xbaadf00d in ?? ()
>   #114 0xbaadf00d in ?? ()
>   #115 0xbaadf00d in ?? ()
>   #116 0xbaadf00d in ?? ()
>   #117 0xbaadf00d in ?? ()
>   #118 0xbaadf00d in ?? ()
>   #119 0xbaadf00d in ?? ()
>   #120 0xbaadf00d in ?? ()
>   #121 0xbaadf00d in ?? ()
>   #122 0xbaadf00d in ?? ()
>   #123 0xbaadf00d in ?? ()
>   #124 0xbaadf00d in ?? ()
>   #125 0xbaadf00d in ?? ()
>   #126 0xbaadf00d in ?? ()
>   #127 0xbaadf00d in ?? ()
>   #128 0xbaadf00d in ?? ()
>   #129 0xbaadf00d in ?? ()
>   #130 0xbaadf00d in ?? ()
>   #131 0xbaadf00d in ?? ()
>   #132 0xbaadf00d in ?? ()
>   #133 0xbaadf00d in ?? ()
>   #134 0xbaadf00d in ?? ()
>   #135 0xbaadf00d in ?? ()
>   #136 0xbaadf00d in ?? ()
>   #137 0xbaadf00d in ?? ()
>   #138 0xbaadf00d in ?? ()
>   #139 0xbaadf00d in ?? ()
>   #140 0xbaadf00d in ?? ()
>   #141 0xbaadf00d in ?? ()
>   #142 0xbaadf00d in ?? ()
>   #143 0xbaadf00d in ?? ()
>   #144 0xbaadf00d in ?? ()
>   #145 0xbaadf00d in ?? ()
>   #146 0xbaadf00d in ?? ()
>   #147 0xbaadf00d in ?? ()
>   #148 0xbaadf00d in ?? ()
>   #149 0xbaadf00d in ?? ()
>   #150 0xbaadf00d in ?? ()
>   #151 0xbaadf00d in ?? ()
>   #152 0xbaadf00d in ?? ()
>   #153 0xbaadf00d in ?? ()
>   #154 0xbaadf00d in ?? ()
>   #155 0xbaadf00d in ?? ()
>   #156 0xbaadf00d in ?? ()
>   #157 0xbaadf00d in ?? ()
>   #158 0xbaadf00d in ?? ()
>   #159 0xbaadf00d in ?? ()
>   #160 0xbaadf00d in ?? ()
>   #161 0xbaadf00d in ?? ()
>   #162 0xbaadf00d in ?? ()
>   #163 0xbaadf00d in ?? ()
>   #164 0xbaadf00d in ?? ()
>   #165 0xbaadf00d in ?? ()
>   #166 0xbaadf00d in ?? ()
>   #167 0xbaadf00d in ?? ()
>   #168 0xbaadf00d in ?? ()
>   #169 0xbaadf00d in ?? ()
>   #170 0xbaadf00d in ?? ()
>   #171 0xbaadf00d in ?? ()
>   #172 0xbaadf00d in ?? ()
>   #173 0xbaadf00d in ?? ()
>   #174 0xbaadf00d in ?? ()
>   #175 0xbaadf00d in ?? ()
>   #176 0xbaadf00d in ?? ()
>   #177 0xbaadf00d in ?? ()
>   #178 0xbaadf00d in ?? ()
>   #179 0xbaadf00d in ?? ()
>   #180 0xbaadf00d in ?? ()
>   #181 0xbaadf00d in ?? ()
>   #182 0xbaadf00d in ?? ()
>   #183 0xbaadf00d in ?? ()
>   #184 0xbaadf00d in ?? ()
>   #185 0xbaadf00d in ?? ()
>   #186 0xbaadf00d in ?? ()
>   #187 0xbaadf00d in ?? ()
>   #188 0xbaadf00d in ?? ()
>   #189 0xbaadf00d in ?? ()
>   #190 0xbaadf00d in ?? ()
>   #191 0xbaadf00d in ?? ()
>   #192 0xbaadf00d in ?? ()
>   #193 0xbaadf00d in ?? ()
>   #194 0xbaadf00d in ?? ()
>   #195 0xbaadf00d in ?? ()
>   #196 0xbaadf00d in ?? ()
>   #197 0xbaadf00d in ?? ()
>   #198 0xbaadf00d in ?? ()
>   #199 0xbaadf00d in ?? ()
>   #200 0xbaadf00d in ?? ()
>   #201 0xbaadf00d in ?? ()
>   #202 0xbaadf00d in ?? ()
>   #203 0xbaadf00d in ?? ()
>   #204 0xbaadf00d in ?? ()
>   #205 0xbaadf00d in ?? ()
>   #206 0xbaadf00d in ?? ()
>   #207 0xbaadf00d in ?? ()
>   #208 0xbaadf00d in ?? ()
>   #209 0xbaadf00d in ?? ()
>   #210 0xbaadf00d in ?? ()
>   #211 0xbaadf00d in ?? ()
>   #212 0xbaadf00d in ?? ()
>   #213 0xbaadf00d in ?? ()
>   #214 0xbaadf00d in ?? ()
>   #215 0xbaadf00d in ?? ()
>   #216 0xbaadf00d in ?? ()
>   #217 0xbaadf00d in ?? ()
>   #218 0xbaadf00d in ?? ()
>   #219 0xbaadf00d in ?? ()
>   #220 0xbaadf00d in ?? ()
>   #221 0xbaadf00d in ?? ()
>   #222 0xbaadf00d in ?? ()
>   #223 0xbaadf00d in ?? ()
>   #224 0xbaadf00d in ?? ()
>   #225 0xbaadf00d in ?? ()
>   #226 0xbaadf00d in ?? ()
>   #227 0xbaadf00d in ?? ()
>   #228 0xbaadf00d in ?? ()
>   #229 0xbaadf00d in ?? ()
>   #230 0xbaadf00d in ?? ()
>   #231 0xbaadf00d in ?? ()
>   #232 0xabababab in ?? ()
>   #233 0xabababab in ?? ()
>   #234 0xfeeefeee in ?? ()
>   #235 0x00000000 in ?? ()
>
> Maybe if I "stepi" inside that last libgccjit function, I will be able
> to advance more.  But something is definitely fishy here, and I'm not
> sure what that is.  Ideas for further debugging are welcome.

Not many here, just two Mr. obvious observations:

Recompiling comp.c at -O0 -g3 might help the broken stepping?  Is SSDATA
(tmp_file) containing something not meaningful or maybe suspicious?

  Andrea