Re: mips system emulation failure with virtio

[Philippe Mathieu-Daudé]

+rth/pm215/dhildenb

On 5/9/23 16:50, Richard Purdie wrote:
> On Tue, 2023-09-05 at 14:59 +0100, Alex Bennée wrote:
> > Richard Purdie <richard.purdie@linuxfoundation.org> writes:
> >
> > > With qemu 8.1.0 we see boot hangs fox x86-64 targets.
> > >
> > > These are fixed by 0d58c660689f6da1e3feff8a997014003d928b3b (softmmu:
> > > Use async_run_on_cpu in tcg_commit) but if I add that commit, mips and
> > > mips64 break, hanging at boot unable to find a rootfs.
> >
> > (Widen CC list)
> >
> > > We use virtio for network and disk and both of those change in the
> > > bootlog from messages like:
> > >
> > > [    1.726118] virtio-pci 0000:00:13.0: enabling device (0000 -> 0003)
> > > [    1.728864] virtio-pci 0000:00:14.0: enabling device (0000 -> 0003)
> > > [    1.729948] virtio-pci 0000:00:15.0: enabling device (0000 -> 0003)
> > > ...
> > > [    2.162148] virtio_blk virtio2: 1/0/0 default/read/poll queues
> > > [    2.168311] virtio_blk virtio2: [vda] 1184242 512-byte logical
> > >
> > > to:
> > >
> > > [    1.777051] virtio-pci 0000:00:13.0: enabling device (0000 -> 0003)
> > > [    1.779822] virtio-pci 0000:00:14.0: enabling device (0000 -> 0003)
> > > [    1.780926] virtio-pci 0000:00:15.0: enabling device (0000 -> 0003)
> > > ...
> > > [    1.894852] virtio_rng: probe of virtio1 failed with error -28
> > > ...
> > > [    2.063553] virtio_blk virtio2: 1/0/0 default/read/poll queues
> > > [    2.064260] virtio_blk: probe of virtio2 failed with error -28
> > > [    2.069080] virtio_net: probe of virtio0 failed with error -28
> > >
> > >
> > > i.e. the virtio drivers no longer work.
> >
> > Interesting, as you say this seems to be VirtIO specific as the baseline
> > tests (using IDE) work fine:
> >
> >    ➜  ./tests/venv/bin/avocado run 
> > ./tests/avocado/tuxrun_baselines.py:test_mips64
> >    JOB ID     : 71f3e3b7080164b78ef1c8c1bb6bc880932d8c9b
> >    JOB LOG    : 
> > /home/alex/avocado/job-results/job-2023-09-05T15.01-71f3e3b/job.log
> >     (1/2) ./tests/avocado/tuxrun_baselines.py:TuxRunBaselineTest.test_mips64: 
> > PASS (12.19 s)
> >     (2/2) ./tests/avocado/tuxrun_baselines.py:TuxRunBaselineTest.test_mips64el: 
> > PASS (11.78 s)
> >    RESULTS    : PASS 2 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | 
> > CANCEL 0
> >    JOB TIME   : 24.79 s
> >
> > > I tested with current qemu master
> > > (17780edd81d27fcfdb7a802efc870a99788bd2fc) and mips is still broken
> > > there.
> > >
> > > Is this issue known about?
> >
> > Could you raise a bug at:
> >
> >    https://gitlab.com/qemu-project/qemu/-/issues
>
> Done, https://gitlab.com/qemu-project/qemu/-/issues/1866
>
> > I'm curious why MIPS VirtIO is affected but nothing else is...
>
> Me too, it seems there is a real code issue somewhere in this...

This seems to fix the issue for me, but I'm not really sure what
I'm doing after various hours debugging, so sharing here before
I take some rest:

-- >8 --
diff --git a/softmmu/physmem.c b/softmmu/physmem.c
index 18277ddd67..ec31ebcb56 100644
--- a/softmmu/physmem.c
+++ b/softmmu/physmem.c
@@ -2517,7 +2517,7 @@ static void tcg_commit(MemoryListener *listener)
      * That said, the listener is also called during realize, before
      * all of the tcg machinery for run-on is initialized: thus halt_cond.
      */
-    if (cpu->halt_cond) {
+    if (cpu->halt_cond && !qemu_cpu_is_self(cpu)) {
         async_run_on_cpu(cpu, tcg_commit_cpu, RUN_ON_CPU_HOST_PTR(cpuas));
     } else {
         tcg_commit_cpu(cpu, RUN_ON_CPU_HOST_PTR(cpuas));
---

That said, the same logic moved generically to async_run_on_cpu()
also works ...:

-- >8 --
diff --git a/cpus-common.c b/cpus-common.c
index 45c745ecf6..b0539c4fb8 100644
--- a/cpus-common.c
+++ b/cpus-common.c
@@ -167,6 +167,9 @@ void do_run_on_cpu(CPUState *cpu, run_on_cpu_func 
func, run_on_cpu_data data,

 void async_run_on_cpu(CPUState *cpu, run_on_cpu_func func, 
run_on_cpu_data data)
 {
+    if (qemu_cpu_is_self(cpu)) {
+        return func(cpu, data);
+    }
     struct qemu_work_item *wi;

     wi = g_new0(struct qemu_work_item, 1);
---

Regards,

Phil.