Re: [RFC PATCH 0/5] hw/arm/virt: Add support for user-creatable nested SMMUv3

[Nathan Chen]

Hi Shameer,

>  Attempt to add the HNS VF to a different SMMUv3 will result in,
>
> -device vfio-pci,host=0000:7d:02.2,bus=pcie.port3,iommufd=iommufd0: 
Unable to attach viommu
> -device vfio-pci,host=0000:7d:02.2,bus=pcie.port3,iommufd=iommufd0: 
vfio 0000:7d:02.2:
>    Failed to set iommu_device: [iommufd=29] error attach 0000:7d:02.2 
(38) to id=11: Invalid argument
>
> At present Qemu is not doing any extra validation other than the above
> failure to make sure the user configuration is correct or not. The
> assumption is libvirt will take care of this.
Would you be able to elaborate what Qemu is validating with this error 
message? I'm not seeing these errors when assigning a GPU's 
pcie-root-port to different PXBs (with different associated SMMU nodes).

I launched a VM using my libvirt prototype code + your qemu branch and 
noted a few small things:
1. Are there plans to support "-device addr" for arm-smmuv3-nested's 
PCIe slot and function like any other device? If not I'll exclude it 
from my libvirt prototype.
2. Is "id" for  "-device arm-smmuv3-nested" necessary for any sort of 
functionality? If so, I'll make a change to my libvirt prototype to 
support this. I was able to boot a VM and see a similar VM PCI topology 
as your example without specifying "id".

Otherwise, the VM topology looks OK with your qemu branch + my libvirt 
prototype.

Also as a heads up, I've added support for auto-inserting PCIe switch 
between the PXB and GPUs in libvirt to attach multiple devices to a SMMU 
node per libvirt's documentation - "If you intend to plug multiple 
devices into a pcie-expander-bus, you must connect a 
pcie-switch-upstream-port to the pcie-root-port that is plugged into the 
pcie-expander-bus, and multiple pcie-switch-downstream-ports to the 
pcie-switch-upstream-port". Future unit-tests should follow this 
topology configuration.

Thanks,
Nathan