Re: KVM_SET_DEVICE_ATTR failed

[Auger Eric]

Hi Zenghui,

On 9/19/20 1:15 PM, Zenghui Yu wrote:
> Hi folks,
> 
> I had booted a guest with an assigned virtual function, with GICv4
> (direct MSI injection) enabled on my arm64 server. I got the following
> QEMU error message on its shutdown:
> 
> "qemu-system-aarch64: KVM_SET_DEVICE_ATTR failed: Group 4 attr
> 0x0000000000000001: Permission denied"
> 
> The problem is that the KVM_DEV_ARM_ITS_SAVE_TABLES ioctl failed while
> stopping the VM.
> 
> As for the kernel side, it turned out that an LPI with irq->hw=true was
> observed while saving ITT for the device. KVM simply failed the save
> operation by returning -EACCES to user-space. The reason is explained in
> the comment block of vgic_its_save_itt(), though I think the HW bit
> should actually be checked in the KVM_DEV_ARM_VGIC_SAVE_PENDING_TABLES
> ioctl rather than in the ITT saving, well, it isn't much related to this
> problem...
> 
> I had noticed that some vectors had been masked by guest VF-driver on
> shutdown, the correspond VLPIs had therefore been unmapped and irq->hw
> was cleared. But some other vectors were un-handled. I *guess* that VFIO
> released these vectors *after* the KVM_DEV_ARM_ITS_SAVE_TABLES ioctl so
> that we end-up trying to save the VLPI's state.
> 
> It may not be a big problem as the guest is going to shutdown anyway and
> the whole guest save/restore on the GICv4.x system is not supported for
> the time being... I'll look at how VFIO would release these vectors but
> post it early in case this is an already known issue (and this might be
> one thing need to be considered if one wants to implement migration on
> the GICv4.x system).

Thanks for reporting the issue. I will have a look at the QEMU sequence too


Eric
> 
> 
> Thanks,
> Zenghui
>