bug#63078: 26.1; Sentinels are sometimes unexpectedly only invoked in script mode

[Markus Triska]

Dear all,

to reproduce this issue, please save the following C code in proc.c:

    #include <stdio.h>
    #include <stdlib.h>
    int main() {
      printf("hello");
      exit(0);
    }

and compile it, yielding the program file "./proc":

    $ gcc proc.c -o proc 

Further, please store the following Elisp definitions in proc.el, in
the same directory as the above program:

    (add-to-list 'exec-path ".")

    (defvar p-stopped nil)

    (defun my-sentinel (proc str)
      (when (string-match "^\\(?:finished\n\\|exited abnormally\\|killed\n\\)"
                          str)
        (setq p-stopped t)))

    (defun wait-for-process-end (proc)
      (while (not p-stopped)
        (message "process status is: %s" (process-status proc))
        (accept-process-output proc 0.1 nil t)))

    (defun f ()
      (with-temp-buffer
        (let ((proc (start-process "p" (current-buffer) "proc")))
          (setq p-stopped nil)
          (set-process-sentinel proc 'my-sentinel)
          (wait-for-process-end proc))
        (message "process said: %s" (buffer-string))))

The following invocation and result shows a case where the program
behaves exactly as intended: It waits for the process sentinel to
signal that no more output can arrive from the process, and shows the
output it got while the process was producing output:

    $ emacs -Q --script proc.el --eval "(f)"
    process status is: exit
    process said: hello

In contrast, the following invocation produces, in the echo area, the
message "process status is: exit", and Emacs hangs unexpectedly:

    $ emacs -Q -l proc.el --eval "(f)"

The behaviour I expect in this case is analogous to the other
invocation: I do expect the message we see, followed by the message
"process said: hello", as in the case above. However, this second
message only appears if I manually interrupt Emacs with C-g.

Regarding the background of this use case:

The goal of one of my Emacs scripts is to reliably obtain *all* output
from a process, until the process ends (which it always does), and
then end the script.

A flawed attempt to do this is to check the state of the process (via
process-status), and end the script when the the state is no longer
'run. However, process output can (and sometimes does) still arrive
even after the state is no longer equal to 'run. Apparently only the
sentinel can reliably detect that no more process output can follow.

Is there a way to make Emacs invoke the sentinel in the example above?

Thank you a lot!
Markus



In GNU Emacs 26.1 (build 3, x86_64-pc-linux-gnu, X toolkit, Xaw scroll bars)
 of 2019-04-09 built on mt-laptop
Windowing system distributor 'The X.Org Foundation', version 11.0.12201003
System Description:     Ubuntu 22.10

Configured using:
 'configure --with-tiff=no'

Configured features:
XPM JPEG GIF PNG SOUND GSETTINGS NOTIFY GNUTLS LIBXML2 FREETYPE XFT ZLIB
TOOLKIT_SCROLL_BARS LUCID X11 THREADS

Important settings:
  value of $LC_MONETARY: en_GB.UTF-8
  value of $LC_NUMERIC: en_GB.UTF-8
  value of $LC_TIME: en_GB.UTF-8
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix