bug#34394: 27.0.50; Emacs segfaults with SLY, company and C-g

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Date: Tue, 12 Feb 2019 20:42:04 +0000
> Cc: 34394@debbugs.gnu.org
> 
> I applied to the emacs-26 branch where it seems to have
> fixed it, meaning I couldn't reproduce it anymore after
> some attempts.
> 
> I would find it prudent to merge into master: since I run
> a master build these days at work, where I use SLY very
> much, it would equal much better testing.
> 
> So I'd push to both branches  (and maybe mark the 26 version
> "don't merge").

Done, thanks for testing and for the reproduction recipe.

> PS: if you have the time, a short explanation of the
> problem would be welcome/pedagogical

Well, remember this question I asked:

> Is it possible that somehow sit_for is called recursively in this
> scenario?  If you put a breakpoint in sit_for, does it break when bt
> shows another instance of sit_for up the call-stack?
> 
> We need to establish who clobbers the jump_buf being used to implement
> quitting by C-g.

You've established that sit_for is not called recursively, but it
turns out read_char is.  It took me a while to find this out, but the
evidence is very clear, see the backtrace at the end of this message.

What happens is this:

  . Emacs becomes idle, calls read_char, and waits for input
  . while waiting, it calls timers
  . one of the timers calls a Company function, which calls your
    completion function
  . your completion function does this:
     . sets up a catch tag
     . defines a function that will throw to that tag
     . sends a string to a subprocess whose process-filter will call
       the above function
     . waits in sit-for, which calls read-event, which calls read_char
  . when the process-filter is invoked, it throws, which makes us end
    up in a call-stack frame _above_ the one of the inner-most call to
    read_char, but without restoring the value of the global
    jmp_buf, which still keeps the value from the now-defunct stack
    frame
  . now you type C-g, and we call longjmp using a bogus jmp_buf

The problematic code was in several places, and looked like this:

          save_getcjmp (save_jump);
          restore_getcjmp (local_getcjmp);
          do_something ();
          restore_getcjmp (save_jump);

save_getcjmp saves the global value of jmp_buf, and restore_getcjmp
restores it.  But the above only works if do_something is guaranteed
to never exit non-locally.  And that guarantee is broken when
do_something calls Lisp, if that Lisp throws.

What my changes do is use an unwind-protect record that ensures the
value of jmp_buf is restored no matter how we exit do_something.

Here's the backtrace I promised that clearly shows the problem.
Noteworthy parts of it:

  . the nested call to read_char
  . the call to Fcatch above the innermost read_char
  . the timer set up by company-mode which started all this

  Breakpoint 3, restore_getcjmp (temp=0x7ffc413393b0) at keyboard.c:3330
  3330    memcpy (getcjmp, temp, sizeof getcjmp);
  #0  restore_getcjmp (temp=0x7ffc413393b0) at keyboard.c:3330
  #1  0x000000000058a033 in read_event_from_main_queue 
(end_time=0x7ffc413395d0, 
      local_getcjmp=0x7ffc413393b0, used_mouse_menu=0x0) at keyboard.c:2148
  #2  0x000000000058a393 in read_decoded_event_from_main_queue (
      end_time=0x7ffc413395d0, local_getcjmp=0x7ffc413393b0, prev_event=XIL(0), 
      used_mouse_menu=0x0) at keyboard.c:2214
  #3  0x000000000058c1cc in read_char (commandflag=0, map=XIL(0), 
      prev_event=XIL(0), used_mouse_menu=0x0, end_time=0x7ffc413395d0)
      at keyboard.c:2802
  #4  0x0000000000676157 in read_filtered_event (no_switch_frame=false, 
      ascii_required=false, error_nonascii=false, input_method=true, 
      seconds=make_number(30)) at lread.c:672
  #5  0x0000000000676471 in Fread_event (prompt=XIL(0), 
      inherit_input_method=XIL(0xbdf0), seconds=make_number(30)) at lread.c:788
  #6  0x0000000000643c97 in funcall_subr (subr=0xd7b760 <Sread_event>, 
      numargs=3, args=0x7ffc41339778) at eval.c:2856
  #7  0x00000000006437ab in Ffuncall (nargs=4, args=0x7ffc41339770)
      at eval.c:2776
  #8  0x000000000069a08e in exec_byte_code (bytestr=XIL(0x9efffc), 
      vector=XIL(0x9f001d), maxdepth=make_number(7), 
      args_template=make_number(769), nargs=1, args=0x7ffc41339bc8)
      at bytecode.c:630
  #9  0x0000000000644389 in funcall_lambda (fun=XIL(0x9effcd), nargs=1, 
      arg_vector=0x7ffc41339bc0) at eval.c:2977
  #10 0x0000000000644018 in apply_lambda (fun=XIL(0x9effcd), 
      args=XIL(0x1690d93), count=51) at eval.c:2913
  #11 0x00000000006421a4 in eval_sub (form=XIL(0x1690da3)) at eval.c:2286
  #12 0x000000000063ea45 in Fwhile (args=XIL(0x1690d83)) at eval.c:989
  #13 0x0000000000641b0e in eval_sub (form=XIL(0x1690e03)) at eval.c:2193
  #14 0x000000000063c90a in Fprogn (body=XIL(0x16902b3)) at eval.c:459
  #15 0x000000000063e3fa in FletX (args=XIL(0x13c8e03)) at eval.c:904
  #16 0x0000000000641b0e in eval_sub (form=XIL(0x13c8df3)) at eval.c:2193
  #17 0x000000000063c90a in Fprogn (body=XIL(0)) at eval.c:459
  #18 0x000000000063edb0 in internal_catch (tag=XIL(0x1027a0), 
      func=0x63c864 <Fprogn>, arg=XIL(0x13d0373)) at eval.c:1101
  #19 0x000000000063ed64 in Fcatch (args=XIL(0x13c8dd3)) at eval.c:1078
  #20 0x0000000000641b0e in eval_sub (form=XIL(0x13c8de3)) at eval.c:2193
  #21 0x000000000063e6c2 in Flet (args=XIL(0x13cf173)) at eval.c:943
  #22 0x0000000000641b0e in eval_sub (form=XIL(0x13cf153)) at eval.c:2193
  #23 0x000000000063c90a in Fprogn (body=XIL(0)) at eval.c:459
  #24 0x000000000064476d in funcall_lambda (fun=XIL(0x13ced13), nargs=0, 
      arg_vector=0x7ffc4133a530) at eval.c:3052
  #25 0x0000000000644018 in apply_lambda (fun=XIL(0x13ced03), args=XIL(0), 
      count=43) at eval.c:2913
  #26 0x00000000006423a2 in eval_sub (form=XIL(0x1688d13)) at eval.c:2316
  #27 0x000000000063e6c2 in Flet (args=XIL(0x168f133)) at eval.c:943
  #28 0x0000000000641b0e in eval_sub (form=XIL(0x168f063)) at eval.c:2193
  #29 0x000000000063c90a in Fprogn (body=XIL(0)) at eval.c:459
  #30 0x000000000064476d in funcall_lambda (fun=XIL(0x1062653), nargs=1, 
      arg_vector=0x7ffc4133aa18) at eval.c:3052
  #31 0x00000000006438e2 in Ffuncall (nargs=2, args=0x7ffc4133aa10)
      at eval.c:2790
  #32 0x0000000000641de4 in eval_sub (form=XIL(0x10fee73)) at eval.c:2212
  #33 0x000000000063c90a in Fprogn (body=XIL(0)) at eval.c:459
  #34 0x0000000000641b0e in eval_sub (form=XIL(0x1104163)) at eval.c:2193
  #35 0x000000000063c6a7 in Fif (args=XIL(0x1104183)) at eval.c:414
  #36 0x0000000000641b0e in eval_sub (form=XIL(0x1104193)) at eval.c:2193
  #37 0x000000000063c90a in Fprogn (body=XIL(0)) at eval.c:459
  #38 0x000000000063e955 in Flet (args=XIL(0x1104203)) at eval.c:973
  #39 0x0000000000641b0e in eval_sub (form=XIL(0x1104213)) at eval.c:2193
  #40 0x000000000063c90a in Fprogn (body=XIL(0)) at eval.c:459
  #41 0x000000000063c803 in Fcond (args=XIL(0x1104263)) at eval.c:439
  #42 0x0000000000641b0e in eval_sub (form=XIL(0x110bcd3)) at eval.c:2193
  #43 0x000000000063c90a in Fprogn (body=XIL(0)) at eval.c:459
  #44 0x000000000064476d in funcall_lambda (fun=XIL(0x110bd83), nargs=2, 
      arg_vector=0x7ffc4133b2b8) at eval.c:3052
  #45 0x00000000006438e2 in Ffuncall (nargs=3, args=0x7ffc4133b2b0)
      at eval.c:2790
  #46 0x0000000000642a2b in Fapply (nargs=2, args=0x7ffc4133b380) at eval.c:2396
  #47 0x0000000000641de4 in eval_sub (form=XIL(0x169fd93)) at eval.c:2212
  #48 0x000000000063c6a7 in Fif (args=XIL(0x169fda3)) at eval.c:414
  #49 0x0000000000641b0e in eval_sub (form=XIL(0x169feb3)) at eval.c:2193
  #50 0x000000000063f7e7 in internal_lisp_condition_case (var=XIL(0x91a140), 
      bodyform=XIL(0x169feb3), handlers=XIL(0x169f413)) at eval.c:1307
  #51 0x000000000063f28b in Fcondition_case (args=XIL(0x169f333)) at eval.c:1231
  #52 0x0000000000641b0e in eval_sub (form=XIL(0x169f323)) at eval.c:2193
  #53 0x000000000063c90a in Fprogn (body=XIL(0)) at eval.c:459
  #54 0x000000000064476d in funcall_lambda (fun=XIL(0x16a6fa3), nargs=2, 
      arg_vector=0x7ffc4133b978) at eval.c:3052
  #55 0x00000000006438e2 in Ffuncall (nargs=3, args=0x7ffc4133b970)
      at eval.c:2790
  #56 0x0000000000642a2b in Fapply (nargs=2, args=0x7ffc4133ba40) at eval.c:2396
  #57 0x0000000000641de4 in eval_sub (form=XIL(0x16a18b3)) at eval.c:2212
  #58 0x000000000063e6c2 in Flet (args=XIL(0x16a00b3)) at eval.c:943
  #59 0x0000000000641b0e in eval_sub (form=XIL(0x16a00a3)) at eval.c:2193
  #60 0x000000000063c90a in Fprogn (body=XIL(0)) at eval.c:459
  #61 0x000000000064476d in funcall_lambda (fun=XIL(0x16a0013), nargs=3, 
      arg_vector=0x7ffc4133be20) at eval.c:3052
  #62 0x0000000000644018 in apply_lambda (fun=XIL(0x16a0003), 
      args=XIL(0x16a1c63), count=23) at eval.c:2913
  #63 0x00000000006423a2 in eval_sub (form=XIL(0x16a1d43)) at eval.c:2316
  #64 0x000000000063c90a in Fprogn (body=XIL(0)) at eval.c:459
  #65 0x000000000064476d in funcall_lambda (fun=XIL(0x16a1973), nargs=2, 
      arg_vector=0x7ffc4133c0c0) at eval.c:3052
  #66 0x0000000000644018 in apply_lambda (fun=XIL(0x16a1963), 
      args=XIL(0x104cc43), count=21) at eval.c:2913
  #67 0x00000000006423a2 in eval_sub (form=XIL(0x104cc13)) at eval.c:2316
  #68 0x0000000000641ee6 in eval_sub (form=XIL(0x104d3d3)) at eval.c:2229
  #69 0x000000000063cbda in Fsetq (args=XIL(0x104d3c3)) at eval.c:517
  #70 0x0000000000641b0e in eval_sub (form=XIL(0x104d3b3)) at eval.c:2193
  #71 0x000000000063c90a in Fprogn (body=XIL(0)) at eval.c:459
  #72 0x000000000063c6f3 in Fif (args=XIL(0x104ce63)) at eval.c:415
  #73 0x0000000000641b0e in eval_sub (form=XIL(0x104ce73)) at eval.c:2193
  #74 0x000000000063c90a in Fprogn (body=XIL(0x104ccc3)) at eval.c:459
  #75 0x000000000063e955 in Flet (args=XIL(0x104cf73)) at eval.c:973
  #76 0x0000000000641b0e in eval_sub (form=XIL(0x104cf83)) at eval.c:2193
  #77 0x000000000063c90a in Fprogn (body=XIL(0)) at eval.c:459
  #78 0x000000000064476d in funcall_lambda (fun=XIL(0x104c833), nargs=0, 
      arg_vector=0x7ffc4133ca08) at eval.c:3052
  #79 0x00000000006438e2 in Ffuncall (nargs=1, args=0x7ffc4133ca00)
      at eval.c:2790
  #80 0x0000000000641de4 in eval_sub (form=XIL(0x12fa1f3)) at eval.c:2212
  #81 0x000000000063cbda in Fsetq (args=XIL(0x12fa203)) at eval.c:517
  #82 0x0000000000641b0e in eval_sub (form=XIL(0x12fa213)) at eval.c:2193
  #83 0x000000000063c90a in Fprogn (body=XIL(0)) at eval.c:459
  #84 0x0000000000641b0e in eval_sub (form=XIL(0x13025d3)) at eval.c:2193
  #85 0x000000000063c6a7 in Fif (args=XIL(0x130cde3)) at eval.c:414
  #86 0x0000000000641b0e in eval_sub (form=XIL(0x130cdc3)) at eval.c:2193
  #87 0x000000000063c90a in Fprogn (body=XIL(0x12f8573)) at eval.c:459
  #88 0x000000000064476d in funcall_lambda (fun=XIL(0x1309753), nargs=1, 
      arg_vector=0x7ffc4133d240) at eval.c:3052
  #89 0x00000000006438e2 in Ffuncall (nargs=2, args=0x7ffc4133d238)
      at eval.c:2790
  #90 0x0000000000642544 in Fapply (nargs=2, args=0x7ffc4133d238) at eval.c:2353
  #91 0x0000000000643b49 in funcall_subr (subr=0xd78e40 <Sapply>, numargs=2, 
      args=0x7ffc4133d238) at eval.c:2831
  #92 0x00000000006437ab in Ffuncall (nargs=3, args=0x7ffc4133d230)
      at eval.c:2776
  #93 0x000000000069a08e in exec_byte_code (bytestr=XIL(0xb04ddc), 
      vector=XIL(0xb04dfd), maxdepth=make_number(10), 
      args_template=make_number(257), nargs=1, args=0x7ffc4133d770)
      at bytecode.c:630
  #94 0x0000000000644389 in funcall_lambda (fun=XIL(0xb04dad), nargs=1, 
      arg_vector=0x7ffc4133d768) at eval.c:2977
  #95 0x00000000006437ef in Ffuncall (nargs=2, args=0x7ffc4133d760)
      at eval.c:2778
  #96 0x00000000006430e4 in call1 (fn=XIL(0xc180), arg1=XIL(0x1662e15))
      at eval.c:2627
  #97 0x0000000000590821 in timer_check_2 (timers=XIL(0), 
      idle_timers=XIL(0x114bd83)) at keyboard.c:4466
  #98 0x000000000059095b in timer_check () at keyboard.c:4528
  #99 0x000000000058df1a in readable_events (flags=1) at keyboard.c:3343
  #100 0x000000000059586d in get_input_pending (flags=1) at keyboard.c:6828
  #101 0x000000000058ff0a in swallow_events (do_display=true) at keyboard.c:4268
  #102 0x00000000004232fd in sit_for (timeout=make_number(30), reading=true, 
      display_option=1) at dispnew.c:5764
  #103 0x000000000058bc87 in read_char (commandflag=1, map=XIL(0x114c013), 
      prev_event=XIL(0), used_mouse_menu=0x7ffc4133dcfd, end_time=0x0)
      at keyboard.c:2717
  #104 0x000000000059b6d9 in read_key_sequence (keybuf=0x7ffc4133dee0, 
      bufsize=30, prompt=XIL(0), dont_downcase_last=false, 
      can_return_switch_frame=true, fix_current_buffer=true, 
      prevent_redisplay=false) at keyboard.c:9151
  #105 0x0000000000587e16 in command_loop_1 () at keyboard.c:1368
  #106 0x000000000063f883 in internal_condition_case (
      bfun=0x5879c0 <command_loop_1>, handlers=XIL(0x4dd0), 
      hfun=0x587008 <cmd_error>) at eval.c:1336
  #107 0x00000000005875ee in command_loop_2 (ignore=XIL(0)) at keyboard.c:1110
  #108 0x000000000063edb0 in internal_catch (tag=XIL(0xc450), 
      func=0x5875c5 <command_loop_2>, arg=XIL(0)) at eval.c:1101
  #109 0x000000000058758e in command_loop () at keyboard.c:1089
  #110 0x0000000000586afd in recursive_edit_1 () at keyboard.c:695
  #111 0x0000000000586ced in Frecursive_edit () at keyboard.c:766
  #112 0x00000000005848fe in main (argc=9, argv=0x7ffc4133e338) at emacs.c:1717

  Lisp Backtrace:
  "read-event" (0x41339778)
  "sit-for" (0x41339bc0)
  "while" (0x41339e08)
  "let*" (0x41339fc8)
  "catch" (0x4133a198)
  "let" (0x4133a388)
  "foo-from-process" (0x4133a530)
  "let" (0x4133a7f8)
  0x1062620 Lisp type 3
  "funcall" (0x4133aa10)
  "progn" (0x4133aba8)
  "if" (0x4133acf8)
  "let" (0x4133af18)
  "cond" (0x4133b0a8)
  "company-capf" (0x4133b2b8)
  "apply" (0x4133b380)
  "if" (0x4133b518)
  "condition-case" (0x4133b768)
  "company-call-backend-raw" (0x4133b978)
  "apply" (0x4133ba40)
  "let" (0x4133bc78)
  "company--force-sync" (0x4133be20)
  "company-call-backend" (0x4133c0c0)
  "cons" (0x4133c2c8)
  "setq" (0x4133c448)
  "if" (0x4133c5c8)
  "let" (0x4133c7e8)
  "company-fetch-metadata" (0x4133ca08)
  "funcall" (0x4133ca00)
  "setq" (0x4133cbc8)
  "progn" (0x4133cd18)
  "if" (0x4133ce68)
  "company-echo-show" (0x4133d240)
  "apply" (0x4133d238)
  "timer-event-handler" (0x4133d768)