Why does calling server-eval-at seem to change the behavior of read-string later?

[Chris Hunt]

Hello,

This is cross-posted from https://emacs.stackexchange.com/q/61676/29147, which
hasn't received much feedback.

I am trying to create a script that, when run with emacs, will read commands
from stdin a line at a time, execute them in another emacs process (with
`server-eval-at`) and print the result. The process should exit when no more
input is available. I have come up with a basic protocol and everything seems
to work when using `eval` locally, but after introducing `server-eval-at` the
emacs process hangs instead of exiting at the end of input.

For initial development I was doing the evaluation inline, in the same process
that was reading the input. That seems to work well. That script looks like

  (while (setq command (ignore-errors (read-string "")))
    (setq command (read command))
    (message "[DEBUG] Received command %s" command)
    (let ((result (eval command)))
      (message "[DEBUG] Response received %s" result)))

In a shell script, for testing:

  #!/bin/sh
  cd "$(mktemp -d)"
  cat <<EOF > script.el
  (while (setq command (ignore-errors (read-string "")))
    (setq command (read command))
    (message "[DEBUG] Received command %s" command)
    (let ((result (eval command)))
      (message "[DEBUG] Response received %s" result)))
  EOF

  echo '(message "hello world")' | \
  emacs --quick --batch --script "$PWD/script.el"

An example of running this, with output:

  $ ./t2
  [DEBUG] Received command (message hello world)
  hello world
  [DEBUG] Response received hello world
  $

As you can see, the debug messages are printed. The process exits at
the end of input.
All of this is expected and OK.

Now I want to run the incoming commands on the server, so I use
`server-eval-at`. Here is that script

  (require 'server)
  (while (setq command (ignore-errors (read-string "")))
    (setq command (read command))
    (message "[DEBUG] Received command %s" command)
    (let ((result (server-eval-at (getenv "RPC_SERVER_NAME") command)))
      (message "[DEBUG] Response received %s" result)))

Another shell script, for testing

  #!/bin/sh
  cd "$(mktemp -d)"

  export RPC_SERVER_NAME="$PWD/emacs"
  emacs --quick "--fg-daemon=$RPC_SERVER_NAME" >stdout.log 2>stderr.log &
  sleep 1

  cat <<EOF > script.el
  (require 'server)
  (while (setq command (ignore-errors (read-string "")))
    (setq command (read command))
    (message "[DEBUG] Received command %s" command)
    (let ((result (server-eval-at (getenv "RPC_SERVER_NAME") command)))
      (message "[DEBUG] Response received %s" result)))
  EOF

  echo '(message "hello world")' | \
  emacs --quick --batch --script "$PWD/script.el"

An example of running this, with output:

  $ ./t1
  [DEBUG] Received command (message hello world)
  [DEBUG] Response received hello world

The output looks fine, but the process does not exit!

I have inspected these processes a few ways:

1. top shows that the `server-eval-at` emacs (client) is consuming 100% CPU.
2. strace shows that the `server-eval-at` emacs (client) is calling `read` on
   stdin (fd 0) repeatedly, and receiving an empty string and 0 result each time
3. strace shows that the `eval` emacs calls `read` on stdin twice: once to read
   the command, and once receiving an empty string and 0 result before exiting.
4. gdb on the `server-eval-at` emacs (client) has a Lisp Backtrace with
   `read-string` at the top.

I inlined `server-eval-at` and was able to identify that calling
[`accept-process-output`](https://github.com/emacs-mirror/emacs/blob/79d04ae13ff33a93f631061d912168e9703251dd/lisp/server.el#L1739)
is necessary for the subsequent call to `read-string` (in the next loop
iteration) to behave differently, but it is hard for me to dig much deeper
than that.

I tried these same steps against emacs 26.3 and observed the same result.

I am using emacs 27.1 on Ubuntu Linux 18.04, as installed with evm.

Any help is appreciated!

Regards,
Chris