Intel IGD passthrough working intermittently

[Marco]

First of all, I'm not really sure where the problem might be, whether
libvirt itself, qemu or some other component; so I'm writing here to
begin the discussion, but please direct me to a more appropriate place
if needed.

I have a machine with an integrated Intel GPU:

00:02.0 VGA compatible controller [0300]: Intel Corporation
CoffeeLake-S GT2 [UHD Graphics 630] [8086:3e91]

That's the only GPU on the machine. I'm doing passthrough of this GPU
to a Windows VM (KVM), with the purpose of getting video output sent
to an external physical display. I'm not interested in host video
output, as the machine is managed via SSH.

After doing all the module blacklisting and vfio-pci binding, and
following the instructions at
https://github.com/qemu/qemu/blob/master/docs/igd-assign.txt, I think
I managed to set up the so-called "legacy" mode.

I'm configuring everything in the grub command line:

vfio-pci.ids=8086:3e91 modprobe.blacklist=i915 intel_iommu=on iommu=pt
pcie_acs_override=downstream,multifunction
initcall_blacklist=sysfb_init video=simplefb:off video=vesafb:off
video=efifb:off video=vesa:off
vfio_iommu_type1.allow_unsafe_interrupts=1 kvm.ignore_msrs=1

(I'm not sure all those options are needed, either)

On Ubuntu 22.04, with libvirt 8.0.0-1ubuntu7.10 and qemu
6.2+dfsg-2ubuntu6.22, I have this domain XML definition:

<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
  ...snip...
  <devices>
    ...snip...
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02'
function='0x0'/>
    </hostdev>
  </devices>
  <qemu:commandline>
    <qemu:arg value='-set'/>
    <qemu:arg value='device.hostdev0.x-vga=on'/>
    <qemu:arg value='-set'/>
    <qemu:arg value='device.hostdev0.x-igd-opregion=on'/>
    <qemu:arg value='-set'/>
    <qemu:arg value='device.hostdev0.x-igd-gms=2'/>
  </qemu:commandline>
</domain>

This generates the following QEMU command line fragment:

-device vfio-pci,host=0000:00:02.0,id=hostdev0,bus=pcie.0,addr=0x2
-set device.hostdev0.x-vga=on -set device.hostdev0.x-igd-opregion=on
-set device.hostdev0.x-igd-gms=2

This works "most" of the times. When it works, I don't see anything
upon VM boot, but as soon as Windows graphics starts, I see output on
the external display.
When it doesn't work, the screen stays black.
In both cases, I see this in dmesg when I start the VM:

[ 1446.153682] vfio-pci 0000:00:02.0: vfio_ecap_init: hiding ecap 0x1b@0x100
[ 1446.340324] vfio-pci 0000:00:02.0: Invalid PCI ROM header
signature: expecting 0xaa55, got 0x0000
[ 1448.162082] vfio-pci 0000:00:02.0: Invalid PCI ROM header
signature: expecting 0xaa55, got 0x0000


I've tried to upgrade QEMU to see whether there would be any change,
using Ubuntu 24.04, with libvirt 10.0.0-2ubuntu8.3 and qemu
8.2.2+ds-0ubuntu1.2. Since I see that libvirt XML syntax for qemu
command override has changed, I've updated the XML as follows:

<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
  ...snip...
  <devices>
    ...snip...
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02'
function='0x0'/>
    </hostdev>
  </devices>
  <qemu:override>
    <qemu:device alias='hostdev0'>
      <qemu:frontend>
        <qemu:property name='x-vga' type='bool' value='true'/>
        <qemu:property name='x-igd-opregion' type='bool' value='true'/>
        <qemu:property name='x-igd-gms' type='signed' value='2'/>
      </qemu:frontend>
    </qemu:device>
  </qemu:override>
</domain>

My understanding is that this should be equivalent to the old
<qemu:arg> syntax I used before.
The generated qemu command line fragment is

-device 
{"driver":"vfio-pci","host":"0000:00:02.0","id":"hostdev0","bus":"pcie.0","addr":"0x2","x-vga":true,"x-igd-opregion":true,"x-igd-gms":2}

But again, this also works only intermittently, in fact I have the
impression that it works less frequently than with the old version of
QEMU (this is of course just an impression; what's certain is that
sometimes it works, sometimes it doesn't).

So I'm trying to see whether this is something I'm doing wrong, some
config I'm missing, or some sort of bug (not sure where). Thanks for
any help.