Changing buffer in filter function has no effect in ERT

[Robin Neatherway]

I have a test for fsharp-mode to exercise the jump to definition
functionality. Communicating with the background process involves
sending a command and then letting the filter function handle the
response, which may involve switching to a different buffer. This
works fine "for real", but no in the test. Strangely, the buffer
switch seems to have worked in the filter function, but has no effect
on what the main test function considers to be the current buffer.

I have created a minimal repro, which is included below. It simply
runs ls, and has a filter function that uses find-file to switch
buffer. The output from message indicates that the buffer switch has
succeeded, but the test function which polls the current buffer at 1s
intervals always outputs the same result for the current buffer.

The test can be run with emacs -Q --batch -l repro.el -l ert -f
ert-run-tests-batch-and-exit if the below text is saved to repro.el.

(require 'ert)

(defun start-proc ()
  (let ((proc (let (process-connection-type)
                (start-process "background" "*background*" "/bin/ls"))))
    (when (process-live-p proc)
      (progn
        (set-process-coding-system proc 'utf-8-auto)
        (set-process-filter proc 'bg-filter-output)
        (set-process-query-on-exit-flag proc nil)))))

(defun bg-filter-output (proc str)
  (message "In filter")
  (find-file "2.txt")
  (message "Buffer name now: %s" (file-name-nondirectory buffer-file-name)))

(defun wait-for-condition (fun)
  (with-timeout (3)
    (while (not (funcall fun))
      (sleep-for 1))))

(ert-deftest change-file-bg ()
  "Check it changes file even in filter output"
  (find-file "1.txt")
  (start-proc)
  (wait-for-condition
   (lambda () (progn
           (message "buffer: %s" (file-name-nondirectory buffer-file-name))
           (not (equal "1.txt" (file-name-nondirectory buffer-file-name))))))
  (should (equal (file-name-nondirectory buffer-file-name) "2.txt")))