[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [PATCH v2 1/1] docs/devel: Add VFIO device migration documentation
From: |
Cornelia Huck |
Subject: |
Re: [PATCH v2 1/1] docs/devel: Add VFIO device migration documentation |
Date: |
Mon, 15 Mar 2021 18:22:58 +0100 |
On Thu, 11 Mar 2021 00:50:09 +0530
Tarun Gupta <targupta@nvidia.com> wrote:
> Document interfaces used for VFIO device migration. Added flow of state
> changes
> during live migration with VFIO device. Tested by building docs with the new
> vfio-migration.rst file.
>
> v2:
> - Included the new vfio-migration.rst file in index.rst
> - Updated dirty page tracking section, also added details about
> 'pre-copy-dirty-page-tracking' opt-out option.
> - Incorporated comments around wording of doc.
>
> Signed-off-by: Tarun Gupta <targupta@nvidia.com>
> Signed-off-by: Kirti Wankhede <kwankhede@nvidia.com>
> ---
> MAINTAINERS | 1 +
> docs/devel/index.rst | 1 +
> docs/devel/vfio-migration.rst | 135 ++++++++++++++++++++++++++++++++++
> 3 files changed, 137 insertions(+)
> create mode 100644 docs/devel/vfio-migration.rst
>
(...)
> diff --git a/docs/devel/vfio-migration.rst b/docs/devel/vfio-migration.rst
> new file mode 100644
> index 0000000000..6196fb132c
> --- /dev/null
> +++ b/docs/devel/vfio-migration.rst
> @@ -0,0 +1,135 @@
> +=====================
> +VFIO device Migration
> +=====================
Maybe add an introductory sentence or two describing the general
approach? I.e. we have a general framework, and specific support for
devices needs to be hooked up.
> +
> +VFIO devices use an iterative approach for migration because certain VFIO
> +devices (e.g. GPU) have large amount of data to be transfered. The iterative
> +pre-copy phase of migration allows for the guest to continue whilst the VFIO
> +device state is transferred to the destination, this helps to reduce the
> total
> +downtime of the VM. VFIO devices can choose to skip the pre-copy phase of
> +migration by returning pending_bytes as zero during the pre-copy phase.
What about something like:
"Migration of VFIO devices consists of two phases: the optional
pre-copy phase, and the stop-and-copy phase. The pre-copy phase is
iterative and allows to accommodate VFIO devices that have a large
amount of data that needs to be transferred. The iterative pre-copy
phase..."
> +
> +A detailed description of the UAPI for VFIO device migration can be found in
> +the comment for the ``vfio_device_migration_info`` structure in the header
> +file linux-headers/linux/vfio.h.
> +
> +VFIO device hooks for iterative approach:
> +
> +* A ``save_setup`` function that sets up the migration region, sets _SAVING
> + flag in the VFIO device state and informs the VFIO IOMMU module to start
> + dirty page tracking.
> +
> +* A ``load_setup`` function that sets up the migration region on the
> + destination and sets _RESUMING flag in the VFIO device state.
> +
> +* A ``save_live_pending`` function that reads pending_bytes from the vendor
> + driver, which indicates the amount of data that the vendor driver has yet
> to
> + save for the VFIO device.
> +
> +* A ``save_live_iterate`` function that reads the VFIO device's data from the
> + vendor driver through the migration region during iterative phase.
> +
> +* A ``save_live_complete_precopy`` function that resets _RUNNING flag from
> the
> + VFIO device state, saves the device config space, if any, and iteratively
> + copies the remaining data for the VFIO device untill the vendor driver
s/untill/until/
> + indicates that no data remains (pending bytes is zero).
> +
> +* A ``load_state`` function that loads the config section and the data
> + sections that are generated by the save functions above
> +
> +* ``cleanup`` functions for both save and load that perform any migration
> + related cleanup, including unmapping the migration region
> +
> +A VM state change handler is registered to change the VFIO device state when
> +the VM state changes.
> +
> +Similarly, a migration state change notifier is registered to get a
> +notification on migration state change. These states are translated to VFIO
s/to/to the corresponding/
> +device state and conveyed to vendor driver.
s/to/to the/
(...)
> +Postcopy
> +========
> +
> +Postcopy migration is not supported for VFIO devices.
s/is not/is currently not/ ?
RE: [PATCH v2 1/1] docs/devel: Add VFIO device migration documentation, Tian, Kevin, 2021/03/11
Re: [PATCH v2 1/1] docs/devel: Add VFIO device migration documentation,
Cornelia Huck <=