KVM_SET_DEVICE_ATTR failed

[Zenghui Yu]

Hi folks,

I had booted a guest with an assigned virtual function, with GICv4
(direct MSI injection) enabled on my arm64 server. I got the following
QEMU error message on its shutdown:

"qemu-system-aarch64: KVM_SET_DEVICE_ATTR failed: Group 4 attr 
0x0000000000000001: Permission denied"

The problem is that the KVM_DEV_ARM_ITS_SAVE_TABLES ioctl failed while
stopping the VM.

As for the kernel side, it turned out that an LPI with irq->hw=true was
observed while saving ITT for the device. KVM simply failed the save
operation by returning -EACCES to user-space. The reason is explained in
the comment block of vgic_its_save_itt(), though I think the HW bit
should actually be checked in the KVM_DEV_ARM_VGIC_SAVE_PENDING_TABLES
ioctl rather than in the ITT saving, well, it isn't much related to this
problem...

I had noticed that some vectors had been masked by guest VF-driver on
shutdown, the correspond VLPIs had therefore been unmapped and irq->hw
was cleared. But some other vectors were un-handled. I *guess* that VFIO
released these vectors *after* the KVM_DEV_ARM_ITS_SAVE_TABLES ioctl so
that we end-up trying to save the VLPI's state.

It may not be a big problem as the guest is going to shutdown anyway and
the whole guest save/restore on the GICv4.x system is not supported for
the time being... I'll look at how VFIO would release these vectors but
post it early in case this is an already known issue (and this might be
one thing need to be considered if one wants to implement migration on
the GICv4.x system).


Thanks,
Zenghui