bug#62194: 30.0.50; Two Eglot-over-Tramp tests are failing on master, passing on emacs-29

[Michael Albinus]

Michael Albinus <michael.albinus@gmx.de> writes:

Hi João,

> This might not be the canonical path, but at least the local tests shall
> find it. I'm running Fedora 37. Could you, pls, check? The log file is 
> appended.

In order to force analysis, I have used as remote server another one,
which runs Ubuntu 22.10. clangd is installed there at a canonical
location:

--8<---------------cut here---------------start------------->8---
# which clangd
/usr/bin/clangd
--8<---------------cut here---------------end--------------->8---

And I run the tests from my Fedora laptop like

--8<---------------cut here---------------start------------->8---
env REMOTE_TEMPORARY_FILE_DIRECTORY='/ssh:albinus@detlef:/tmp' make -C test 
eglot-tests.log SELECTOR='"tramp"'
--8<---------------cut here---------------end--------------->8---

Sometimes the tests pass, sometimes not. In the latter case, it is the
same picture as you have reported. But I wanted to reproduce it locally.

The most interesting part is the backtrace:

--8<---------------cut here---------------start------------->8---
Test eglot-test-tramp-test backtrace:
  tramp-sh-handle-file-attributes("/ssh:albinus@detlef:/tmp/eglot--fix
  apply(tramp-sh-handle-file-attributes "/ssh:albinus@detlef:/tmp/eglo
  tramp-vc-file-name-handler(file-attributes "/ssh:albinus@detlef:/tmp
  file-attributes("/ssh:albinus@detlef:/tmp/eglot--fixture2kRNPt/proj.
  tramp-handle-file-symlink-p("/ssh:albinus@detlef:/tmp/eglot--fixture
  apply(tramp-handle-file-symlink-p "/ssh:albinus@detlef:/tmp/eglot--f
  tramp-vc-file-name-handler(file-symlink-p "/ssh:albinus@detlef:/tmp/
  file-symlink-p("/ssh:albinus@detlef:/tmp/eglot--fixture2kRNPt/proj..
  tramp-sh-handle-file-truename("/ssh:albinus@detlef:/tmp/eglot--fixtu
  apply(tramp-sh-handle-file-truename "/ssh:albinus@detlef:/tmp/eglot-
  tramp-vc-file-name-handler(file-truename "/ssh:albinus@detlef:/tmp/e
  file-truename("/ssh:albinus@detlef:/tmp/eglot--fixture2kRNPt/proj...
  find-buffer-visiting("/ssh:albinus@detlef:/tmp/eglot--fixture2kRNPt/
  #f(compiled-function (arg1 arg2 &rest rest) "Handle notification pub
  apply(#f(compiled-function (arg1 arg2 &rest rest) "Handle notificati
  eglot-handle-notification(#<eglot-lsp-server eglot-lsp-server-9d34aa
  apply(eglot-handle-notification #<eglot-lsp-server eglot-lsp-server-
  #f(compiled-function (server method params) #<bytecode -0xa39641d878
  jsonrpc-connection-receive(#<eglot-lsp-server eglot-lsp-server-9d34a
  jsonrpc--process-filter(#<process EGLOT (project/(c-mode c-ts-mode c
  accept-process-output(#<process EGLOT (project/(c-mode c-ts-mode c++
  tramp-accept-process-output(#<process *tramp/ssh albinus@detlef*>)
  tramp-wait-for-regexp(#<process *tramp/ssh albinus@detlef*> nil "\\(
  tramp-wait-for-output(#<process *tramp/ssh albinus@detlef*>)
  tramp-send-command((tramp-file-name "ssh" "albinus" nil "detlef" nil
  tramp-send-command-and-check((tramp-file-name "ssh" "albinus" nil "d
  tramp-run-test("-d" "/ssh:albinus@detlef:/tmp/eglot--fixture2kRNPt")
  tramp-sh-handle-file-directory-p("/ssh:albinus@detlef:/tmp/eglot--fi
  apply(tramp-sh-handle-file-directory-p "/ssh:albinus@detlef:/tmp/egl
  tramp-vc-file-name-handler(file-directory-p "/ssh:albinus@detlef:/tm
  file-directory-p("/ssh:albinus@detlef:/tmp/eglot--fixture2kRNPt/")
  locate-dominating-file("/ssh:albinus@detlef:/tmp/eglot--fixture2kRNP
  vc-find-root("/ssh:albinus@detlef:/tmp/eglot--fixture2kRNPt/proj..."
  vc-svn-registered("/ssh:albinus@detlef:/tmp/eglot--fixture2kRNPt/pro
  apply(vc-svn-registered "/ssh:albinus@detlef:/tmp/eglot--fixture2kRN
  vc-call-backend(SVN registered "/ssh:albinus@detlef:/tmp/eglot--fixt
  #f(compiled-function (b) #<bytecode 0x133be07a00af7442>)(SVN)
  mapc(#f(compiled-function (b) #<bytecode 0x133be07a00af7442>) (RCS C
  vc-registered("/ssh:albinus@detlef:/tmp/eglot--fixture2kRNPt/proj...
  apply(vc-registered "/ssh:albinus@detlef:/tmp/eglot--fixture2kRNPt/p
  tramp-run-real-handler(vc-registered ("/ssh:albinus@detlef:/tmp/eglo
  tramp-sh-handle-vc-registered("/ssh:albinus@detlef:/tmp/eglot--fixtu
  apply(tramp-sh-handle-vc-registered "/ssh:albinus@detlef:/tmp/eglot-
  tramp-sh-file-name-handler(vc-registered "/ssh:albinus@detlef:/tmp/e
  apply(tramp-sh-file-name-handler vc-registered "/ssh:albinus@detlef:
  tramp-file-name-handler(vc-registered "/ssh:albinus@detlef:/tmp/eglo
  vc-registered("/ssh:albinus@detlef:/tmp/eglot--fixture2kRNPt/proj...
  vc-backend("/ssh:albinus@detlef:/tmp/eglot--fixture2kRNPt/proj...")
  vc-refresh-state()
  run-hooks(find-file-hook)
  after-find-file(nil t)
  find-file-noselect-1(#<killed buffer> "/ssh:albinus@detlef:/tmp/eglo
  find-file-noselect("project/merdix.c")
  eglot--find-file-noselect("project/merdix.c")
  (set-buffer (eglot--find-file-noselect "project/merdix.c"))
  (save-current-buffer (set-buffer (eglot--find-file-noselect "project
  (closure ((server . #<eglot-lsp-server eglot-lsp-server-9d34aa>)) ni
  funcall((closure ((server . #<eglot-lsp-server eglot-lsp-server-9d34
  (prog1 (funcall fn) (setq test-body-successful-p t))
  (let* ((process-environment (append (list "XDG_CONFIG_HOME=/dev/null
  (unwind-protect (let* ((process-environment (append (list "XDG_CONFI
  (let* ((fixture-directory (make-nearby-temp-file "eglot--fixture" t)
  eglot--call-with-fixture((("project" ("coiso.c" . "bla") ("merdix.c"
  (let (server) (eglot--call-with-fixture '(("project" ("coiso.c" . "b
  eglot-tests--auto-detect-running-server-1()
  funcall(eglot-tests--auto-detect-running-server-1)
  (let* ((tramp-remote-path (cons 'tramp-own-remote-path tramp-remote-
  eglot--call-with-tramp-test(eglot-tests--auto-detect-running-server-
  (closure (tramp-histfile-override company-candidates typescript-mode
  ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
  ert-run-test(#s(ert-test :name eglot-test-tramp-test :documentation 
  ert-run-or-rerun-test(#s(ert--stats :selector "tramp" :tests ... :te
  ert-run-tests("tramp" #f(compiled-function (event-type &rest event-a
  ert-run-tests-batch("tramp")
  ert-run-tests-batch-and-exit("tramp")
  eval((ert-run-tests-batch-and-exit '"tramp") t)
  command-line-1(("-L" ":." "-l" "ert" "-l" "lisp/progmodes/eglot-test
  command-line()
  normal-top-level()
--8<---------------cut here---------------end--------------->8---

Let's analyze bottom up. The very first time Tramp is involved is here:

--8<---------------cut here---------------start------------->8---
  tramp-file-name-handler(vc-registered "/ssh:albinus@detlef:/tmp/eglo
  vc-registered("/ssh:albinus@detlef:/tmp/eglot--fixture2kRNPt/proj...
  vc-backend("/ssh:albinus@detlef:/tmp/eglot--fixture2kRNPt/proj...")
  vc-refresh-state()
  run-hooks(find-file-hook)
  after-find-file(nil t)
  find-file-noselect-1(#<killed buffer> "/ssh:albinus@detlef:/tmp/eglo
  find-file-noselect("project/merdix.c")
  eglot--find-file-noselect("project/merdix.c")
--8<---------------cut here---------------end--------------->8---

That is, file "project/merdix.c" is viewed, which has the absolute path
"/ssh:albinus@detlef:/tmp/eglot--fixture2kRNPt/project/merdix.c". It has
been loaded into the buffer successfully, and 'run-hooks(find-file-hook)'
is applied. Since the file is under vc control, 'vc-registerd' is invoked.
Some lines up in the backtrace, we see

--8<---------------cut here---------------start------------->8---
  jsonrpc--process-filter(#<process EGLOT (project/(c-mode c-ts-mode c
  accept-process-output(#<process EGLOT (project/(c-mode c-ts-mode c++
  tramp-accept-process-output(#<process *tramp/ssh albinus@detlef*>)
  tramp-wait-for-regexp(#<process *tramp/ssh albinus@detlef*> nil "\\(
  tramp-wait-for-output(#<process *tramp/ssh albinus@detlef*>)
  tramp-send-command((tramp-file-name "ssh" "albinus" nil "detlef" nil
  tramp-send-command-and-check((tramp-file-name "ssh" "albinus" nil "d
  tramp-run-test("-d" "/ssh:albinus@detlef:/tmp/eglot--fixture2kRNPt")
  tramp-sh-handle-file-directory-p("/ssh:albinus@detlef:/tmp/eglot--fi
--8<---------------cut here---------------end--------------->8---

Tramp needs to know, whether a remote file is a directory, and it
sends the command "test -d /tmp/eglot--fixture2kRNPt". It waits for the
result in 'tramp-accept-process-output(#<process *tramp/ssh albinus@detlef*>)'
However, 'tramp-accept-process-output' reads also the jsonrpc output by 
'accept-process-output(#<process EGLOT (project/(c-mode c-ts-mode c++...',
which triggers 'jsonrpc--process-filter'. This results finally, on top of
the backtrace, in another remote Tramp command:

--8<---------------cut here---------------start------------->8---
  tramp-sh-handle-file-attributes("/ssh:albinus@detlef:/tmp/eglot--fix
  apply(tramp-sh-handle-file-attributes "/ssh:albinus@detlef:/tmp/eglo
  tramp-vc-file-name-handler(file-attributes "/ssh:albinus@detlef:/tmp
  file-attributes("/ssh:albinus@detlef:/tmp/eglot--fixture2kRNPt/proj.
--8<---------------cut here---------------end--------------->8---

It isn't visible in the backtrace, but it is for sure the famous
"Forbidden reentrant call of Tramp" error.

So I reommend the following:

- At least in 'eglot--call-with-tramp-test', better in your Eglot
  package, suppress vc related checks, by binding

--8<---------------cut here---------------start------------->8---
  (let ((vc-ignore-dir-regexp
         (format "\\(%s\\)\\|\\(%s\\)"
                 vc-ignore-dir-regexp
                 tramp-file-name-regexp)))
     ...)
--8<---------------cut here---------------end--------------->8---

- If this works sufficiently, this bug shall be either closed, or merged
  with bug#60534, which tracks this error.

Best regards, Michael.