On Thu, 8 May 2025 at 14:46, Donald Dutile <ddutile@redhat.com> wrote:
I would refer to the ARM SMMU spec, Figure 2.3 in the G.a version, where
it's slightly different; more like:
+------------------+
| PCIe Devices | (one device, unless a PCIe switch is btwn the RC &
'Devices';
+------------------+ or, see more typical expansion below)
|
+-------------+
| PCIe RC A |
+-------------+
|
+------v---+ +-----------------------------------+
| SMMUv3.A | | Wide assortment of other platform |
| (IOMMU) | | devices not using SMMU |
+----+-----+ +-----------------------------------+
| | | |
+------+----------------------+---+---+-+
| System Interconnect |
+---------------------------------------+
|
+-------+--------+ +-----+-------------+
| System RAM |<--->| CPU (NUMA socket) |
+----------------+ +-------------------+
In fact, the PCIe can be quite complex with PCIe bridges, and multiple Root
Ports (RP's),
and multiple SMMU's:
+--------------+ +--------------+ +--------------+
| PCIe Device | | PCIe Device | | PCIe Device |
+--------------+ +--------------+ +--------------+
| | | <--- PCIe bus
+----------+ +----------+ +----------+
| PCIe RP | | PCIe RP | | PCIe RP | <- may be PCI Bridge,
may not
+----------+ +----------+ +----------+
| | |
+----------+ +----------+ +----------+
| SMMU | | SMMU | | SMMU |
+----------+ +----------+ +----------+
| | | <- may be a bus, may
not(hidden from OS)
+------------------+------------------+
|
+--------------------------+
| PCI RC A |
+--------------------------+
The final take away: the (QEMU) SMMU/IOMMU must be associated with a PCIe bus
OR, the format has to be something like:
-device smmuv3, id=smmuv3.1
-device <blah>, smmu=smmuv3.1
where the device <-> SMMU (or if extended to x86, iommu) associativity is set
w/o bus associativity.
The problem here seems to me to be that in the hardware we're
modelling the SMMU always exists, because it's in the SoC,
but you're trying to arrange for it to be created on the
command line, via -device.
We don't have any of these problems with the current 'virt'
board code, because we have the board code create the iommu
(if the user asks for it via the iommu machine property),
and it can wire it up to the PCI root complex as needed.
thanks
-- PMM