[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[PATCH v4 14/15] vfio/common: Optimize device dirty page tracking with v
From: |
Joao Martins |
Subject: |
[PATCH v4 14/15] vfio/common: Optimize device dirty page tracking with vIOMMU |
Date: |
Thu, 22 Jun 2023 22:48:44 +0100 |
From: Avihai Horon <avihaih@nvidia.com>
When vIOMMU is enabled, syncing dirty page bitmaps is done by replaying
the vIOMMU mappings and querying the dirty bitmap for each mapping.
With device dirty tracking this causes a lot of overhead, since the HW
is queried many times (even with small idle guest this can end up with
thousands of calls to HW).
Optimize this by de-coupling dirty bitmap query from vIOMMU replay.
Now a single dirty bitmap is queried per vIOMMU MR section, which is
then used for all corresponding vIOMMU mappings within that MR section.
Signed-off-by: Avihai Horon <avihaih@nvidia.com>
Signed-off-by: Joao Martins <joao.m.martins@oracle.com>
---
hw/vfio/common.c | 74 ++++++++++++++++++++++++++++++++++++++++++++++--
1 file changed, 72 insertions(+), 2 deletions(-)
diff --git a/hw/vfio/common.c b/hw/vfio/common.c
index c530e9d87f21..62f91e8e102d 100644
--- a/hw/vfio/common.c
+++ b/hw/vfio/common.c
@@ -1832,8 +1832,36 @@ out:
typedef struct {
IOMMUNotifier n;
VFIOGuestIOMMU *giommu;
+ VFIOBitmap vbmap;
} vfio_giommu_dirty_notifier;
+static int vfio_iommu_set_dirty_bitmap(VFIOContainer *container,
+ vfio_giommu_dirty_notifier *gdn,
+ hwaddr iova, hwaddr size,
+ ram_addr_t ram_addr)
+{
+ VFIOBitmap *vbmap = &gdn->vbmap;
+ VFIOBitmap dst_vbmap;
+ hwaddr start_iova = REAL_HOST_PAGE_ALIGN(gdn->n.start);
+ hwaddr copy_offset;
+ int ret;
+
+ ret = vfio_bitmap_alloc(&dst_vbmap, size);
+ if (ret) {
+ return -ENOMEM;
+ }
+
+ copy_offset = (iova - start_iova) / qemu_real_host_page_size();
+ bitmap_copy_with_src_offset(dst_vbmap.bitmap, vbmap->bitmap, copy_offset,
+ dst_vbmap.pages);
+
+ cpu_physical_memory_set_dirty_lebitmap(dst_vbmap.bitmap, ram_addr,
+ dst_vbmap.pages);
+ g_free(dst_vbmap.bitmap);
+
+ return 0;
+}
+
static void vfio_iommu_map_dirty_notify(IOMMUNotifier *n, IOMMUTLBEntry *iotlb)
{
vfio_giommu_dirty_notifier *gdn = container_of(n,
@@ -1854,8 +1882,15 @@ static void vfio_iommu_map_dirty_notify(IOMMUNotifier
*n, IOMMUTLBEntry *iotlb)
rcu_read_lock();
if (vfio_get_xlat_addr(iotlb, NULL, &translated_addr, NULL)) {
- ret = vfio_get_dirty_bitmap(container, iova, iotlb->addr_mask + 1,
- translated_addr);
+ if (gdn->vbmap.bitmap) {
+ ret = vfio_iommu_set_dirty_bitmap(container, gdn, iova,
+ iotlb->addr_mask + 1,
+ translated_addr);
+ } else {
+ ret = vfio_get_dirty_bitmap(container, iova, iotlb->addr_mask + 1,
+ translated_addr);
+ }
+
if (ret) {
error_report("vfio_iommu_map_dirty_notify(%p, 0x%"HWADDR_PRIx", "
"0x%"HWADDR_PRIx") = %d (%s)",
@@ -1936,6 +1971,7 @@ static int vfio_sync_iommu_dirty_bitmap(VFIOContainer
*container,
}
gdn.giommu = giommu;
+ gdn.vbmap.bitmap = NULL;
idx = memory_region_iommu_attrs_to_index(giommu->iommu_mr,
MEMTXATTRS_UNSPECIFIED);
@@ -1943,10 +1979,44 @@ static int vfio_sync_iommu_dirty_bitmap(VFIOContainer
*container,
section->size);
llend = int128_sub(llend, int128_one());
+ /*
+ * Optimize device dirty tracking if the MR section is at least partially
+ * tracked. Optimization is done by querying a single dirty bitmap for the
+ * entire range instead of querying dirty bitmap for each vIOMMU mapping.
+ */
+ if (vfio_devices_all_device_dirty_tracking(container)) {
+ hwaddr start = REAL_HOST_PAGE_ALIGN(section->offset_within_region);
+ hwaddr end = int128_get64(llend);
+ hwaddr iommu_max_iova;
+ hwaddr size;
+ int ret;
+
+ ret = vfio_viommu_get_max_iova(&iommu_max_iova);
+ if (ret) {
+ return -EINVAL;
+ }
+
+ size = REAL_HOST_PAGE_ALIGN(MIN(iommu_max_iova, end) - start);
+
+ ret = vfio_bitmap_alloc(&gdn.vbmap, size);
+ if (ret) {
+ return -ENOMEM;
+ }
+
+ ret = vfio_devices_query_dirty_bitmap(container, &gdn.vbmap,
+ start, size);
+ if (ret) {
+ g_free(gdn.vbmap.bitmap);
+
+ return ret;
+ }
+ }
+
iommu_notifier_init(&gdn.n, vfio_iommu_map_dirty_notify,
IOMMU_NOTIFIER_MAP,
section->offset_within_region, int128_get64(llend),
idx);
memory_region_iommu_replay(giommu->iommu_mr, &gdn.n);
+ g_free(gdn.vbmap.bitmap);
return 0;
}
--
2.17.2
- [PATCH v4 02/15] hw/pci: Refactor pci_device_iommu_address_space(), (continued)
- [PATCH v4 02/15] hw/pci: Refactor pci_device_iommu_address_space(), Joao Martins, 2023/06/22
- [PATCH v4 03/15] hw/pci: Introduce pci_device_iommu_get_attr(), Joao Martins, 2023/06/22
- [PATCH v4 04/15] intel-iommu: Switch to pci_setup_iommu_ops(), Joao Martins, 2023/06/22
- [PATCH v4 05/15] memory/iommu: Add IOMMU_ATTR_DMA_TRANSLATION attribute, Joao Martins, 2023/06/22
- [PATCH v4 06/15] intel-iommu: Implement get_attr() method, Joao Martins, 2023/06/22
- [PATCH v4 07/15] vfio/common: Track whether DMA Translation is enabled on the vIOMMU, Joao Martins, 2023/06/22
- [PATCH v4 08/15] vfio/common: Relax vIOMMU detection when DMA translation is off, Joao Martins, 2023/06/22
- [PATCH v4 09/15] memory/iommu: Add IOMMU_ATTR_MAX_IOVA attribute, Joao Martins, 2023/06/22
- [PATCH v4 10/15] intel-iommu: Implement IOMMU_ATTR_MAX_IOVA get_attr() attribute, Joao Martins, 2023/06/22
- [PATCH v4 11/15] vfio/common: Move dirty tracking ranges update to helper, Joao Martins, 2023/06/22
- [PATCH v4 14/15] vfio/common: Optimize device dirty page tracking with vIOMMU,
Joao Martins <=
- [PATCH v4 13/15] vfio/common: Extract vIOMMU code from vfio_sync_dirty_bitmap(), Joao Martins, 2023/06/22
- [PATCH v4 15/15] vfio/common: Block migration with vIOMMUs without address width limits, Joao Martins, 2023/06/22
- [PATCH v4 12/15] vfio/common: Support device dirty page tracking with vIOMMU, Joao Martins, 2023/06/22
- Re: [PATCH v4 00/15] vfio: VFIO migration support with vIOMMU, Joao Martins, 2023/06/22