[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Qemu-devel] [PULL v2 13/14] vhost-scsi: Allow user to enable migration
From: |
Michael S. Tsirkin |
Subject: |
[Qemu-devel] [PULL v2 13/14] vhost-scsi: Allow user to enable migration |
Date: |
Mon, 3 Jun 2019 14:09:07 -0400 |
From: Liran Alon <address@hidden>
In order to perform a valid migration of a vhost-scsi device,
the following requirements must be met:
(1) The virtio-scsi device state needs to be saved & loaded.
(2) The vhost backend must be stopped before virtio-scsi device state
is saved:
(2.1) Sync vhost backend state to virtio-scsi device state.
(2.2) No further I/O requests are made by vhost backend to target
SCSI device.
(2.3) No further guest memory access takes place after VM is stopped.
(3) Requests in-flight to target SCSI device are completed before
migration handover.
(4) Target SCSI device state needs to be saved & loaded into the
destination host target SCSI device.
Previous commit ("vhost-scsi: Add VMState descriptor")
add support to save & load the device state using VMState.
This meets requirement (1).
When VM is stopped by migration thread (On Pre-Copy complete), the
following code path is executed:
migration_completion() -> vm_stop_force_state() -> vm_stop() ->
do_vm_stop().
do_vm_stop() calls first pause_all_vcpus() which pause all guest
vCPUs and then call vm_state_notify().
In case of vhost-scsi device, this will lead to the following code path
to be executed:
vm_state_notify() -> virtio_vmstate_change() ->
virtio_set_status() -> vhost_scsi_set_status() -> vhost_scsi_stop().
vhost_scsi_stop() then calls vhost_scsi_clear_endpoint() and
vhost_scsi_common_stop().
vhost_scsi_clear_endpoint() sends VHOST_SCSI_CLEAR_ENDPOINT ioctl to
vhost backend which will reach kernel's vhost_scsi_clear_endpoint()
which process all pending I/O requests and wait for them to complete
(vhost_scsi_flush()). This meets requirement (3).
vhost_scsi_common_stop() will stop the vhost backend.
As part of this stop, dirty-bitmap is synced and vhost backend state is
synced with virtio-scsi device state. As at this point guest vCPUs are
already paused, this meets requirement (2).
At this point we are left with requirement (4) which is target SCSI
device specific and therefore cannot be done by QEMU. Which is the main
reason why vhost-scsi adds a migration blocker.
However, as this can be handled either by an external orchestrator or
by using shared-storage (i.e. iSCSI), there is no reason to limit the
orchestrator from being able to explictly specify it wish to enable
migration even when VM have a vhost-scsi device.
Considering all the above, this commit allows orchestrator to explictly
specify that it is responsbile for taking care of requirement (4) and
therefore vhost-scsi should not add a migration blocker.
Reviewed-by: Nir Weiner <address@hidden>
Reviewed-by: Bijan Mottahedeh <address@hidden>
Signed-off-by: Liran Alon <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Michael S. Tsirkin <address@hidden>
Signed-off-by: Michael S. Tsirkin <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
---
include/hw/virtio/vhost-scsi-common.h | 1 +
hw/scsi/vhost-scsi.c | 30 ++++++++++++++++++---------
2 files changed, 21 insertions(+), 10 deletions(-)
diff --git a/include/hw/virtio/vhost-scsi-common.h
b/include/hw/virtio/vhost-scsi-common.h
index 57fb1d87b5..4eab767ee8 100644
--- a/include/hw/virtio/vhost-scsi-common.h
+++ b/include/hw/virtio/vhost-scsi-common.h
@@ -36,6 +36,7 @@ typedef struct VHostSCSICommon {
int target;
int lun;
uint64_t host_features;
+ bool migratable;
} VHostSCSICommon;
int vhost_scsi_common_start(VHostSCSICommon *vsc);
diff --git a/hw/scsi/vhost-scsi.c b/hw/scsi/vhost-scsi.c
index eb0cf9e131..6b01accf61 100644
--- a/hw/scsi/vhost-scsi.c
+++ b/hw/scsi/vhost-scsi.c
@@ -199,13 +199,18 @@ static void vhost_scsi_realize(DeviceState *dev, Error
**errp)
goto close_fd;
}
- error_setg(&vsc->migration_blocker,
- "vhost-scsi does not support migration");
- migrate_add_blocker(vsc->migration_blocker, &err);
- if (err) {
- error_propagate(errp, err);
- error_free(vsc->migration_blocker);
- goto close_fd;
+ if (!vsc->migratable) {
+ error_setg(&vsc->migration_blocker,
+ "vhost-scsi does not support migration in all cases. "
+ "When external environment supports it (Orchestrator migrates "
+ "target SCSI device state or use shared storage over network),
"
+ "set 'migratable' property to true to enable migration.");
+ migrate_add_blocker(vsc->migration_blocker, &err);
+ if (err) {
+ error_propagate(errp, err);
+ error_free(vsc->migration_blocker);
+ goto close_fd;
+ }
}
vsc->dev.nvqs = VHOST_SCSI_VQ_NUM_FIXED + vs->conf.num_queues;
@@ -230,7 +235,9 @@ static void vhost_scsi_realize(DeviceState *dev, Error
**errp)
return;
free_vqs:
- migrate_del_blocker(vsc->migration_blocker);
+ if (!vsc->migratable) {
+ migrate_del_blocker(vsc->migration_blocker);
+ }
g_free(vsc->dev.vqs);
close_fd:
close(vhostfd);
@@ -243,8 +250,10 @@ static void vhost_scsi_unrealize(DeviceState *dev, Error
**errp)
VHostSCSICommon *vsc = VHOST_SCSI_COMMON(dev);
struct vhost_virtqueue *vqs = vsc->dev.vqs;
- migrate_del_blocker(vsc->migration_blocker);
- error_free(vsc->migration_blocker);
+ if (!vsc->migratable) {
+ migrate_del_blocker(vsc->migration_blocker);
+ error_free(vsc->migration_blocker);
+ }
/* This will stop vhost backend. */
vhost_scsi_set_status(vdev, 0);
@@ -268,6 +277,7 @@ static Property vhost_scsi_properties[] = {
DEFINE_PROP_BIT64("t10_pi", VHostSCSICommon, host_features,
VIRTIO_SCSI_F_T10_PI,
false),
+ DEFINE_PROP_BOOL("migratable", VHostSCSICommon, migratable, false),
DEFINE_PROP_END_OF_LIST(),
};
--
MST
- [Qemu-devel] [PULL v2 04/14] pci: Make is_bridge a bool, (continued)
- [Qemu-devel] [PULL v2 09/14] vhost: fix memory leak in vhost_user_scsi_realize, Michael S. Tsirkin, 2019/06/03
- [Qemu-devel] [PULL v2 07/14] vhost: remove the dead code, Michael S. Tsirkin, 2019/06/03
- [Qemu-devel] [PULL v2 06/14] docs: smbios: remove family=x from type2 entry description, Michael S. Tsirkin, 2019/06/03
- [Qemu-devel] [PULL v2 10/14] bios-tables-test: add diff allowed list, Michael S. Tsirkin, 2019/06/03
- [Qemu-devel] [PULL v2 11/14] vhost-scsi: The vhost backend should be stopped when the VM is not running, Michael S. Tsirkin, 2019/06/03
- [Qemu-devel] [PULL v2 12/14] vhost-scsi: Add VMState descriptor, Michael S. Tsirkin, 2019/06/03
- [Qemu-devel] [PULL v2 13/14] vhost-scsi: Allow user to enable migration,
Michael S. Tsirkin <=
- [Qemu-devel] [PULL v2 14/14] bios-tables-test: list all tables that differ, Michael S. Tsirkin, 2019/06/03
- Re: [Qemu-devel] [PULL v2 00/14] virtio, pci, pc: cleanups, features, Peter Maydell, 2019/06/04
- Re: [Qemu-devel] [PULL v2 00/14] virtio, pci, pc: cleanups, features, Michael S. Tsirkin, 2019/06/04
- Re: [Qemu-devel] [PULL v2 00/14] virtio, pci, pc: cleanups, features, Peter Maydell, 2019/06/04
- Re: [Qemu-devel] [PULL v2 00/14] virtio, pci, pc: cleanups, features, Michael S. Tsirkin, 2019/06/04