[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [qemu-web PATCH] Add virtio-blk and virtio-scsi configuration post
|
From: |
Daniel P . Berrangé |
|
Subject: |
Re: [qemu-web PATCH] Add virtio-blk and virtio-scsi configuration post |
|
Date: |
Tue, 15 Sep 2020 11:55:01 +0100 |
|
User-agent: |
Mutt/1.14.6 (2020-07-11) |
On Tue, Sep 15, 2020 at 11:34:06AM +0100, Stefan Hajnoczi wrote:
> The second post in the storage series covers virtio-blk and virtio-scsi.
> It compares the two and offers recommendations that users and tools
> using QEMU can use as a starting point. Graphs are included comparing
> the performance of various options.
>
> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
> ---
> ...020-09-15-virtio-blk-scsi-configuration.md | 112 ++++++++++++++++++
> screenshots/2020-09-15-scsi-devices.svg | 1 +
> screenshots/2020-09-15-virtio-blk-vs-scsi.svg | 1 +
> 3 files changed, 114 insertions(+)
> create mode 100644 _posts/2020-09-15-virtio-blk-scsi-configuration.md
> create mode 100644 screenshots/2020-09-15-scsi-devices.svg
> create mode 100644 screenshots/2020-09-15-virtio-blk-vs-scsi.svg
>
> diff --git a/_posts/2020-09-15-virtio-blk-scsi-configuration.md
> b/_posts/2020-09-15-virtio-blk-scsi-configuration.md
> new file mode 100644
> index 0000000..0ba9c87
> --- /dev/null
> +++ b/_posts/2020-09-15-virtio-blk-scsi-configuration.md
> @@ -0,0 +1,112 @@
> +---
> +layout: post
> +title: "Configuring virtio-blk and virtio-scsi Devices"
> +date: 2020-09-15 07:00:00 +0000
> +author: Stefan Hajnoczi and Sergio Lopez
> +categories: [storage]
> +---
> +The [previous
> article](https://www.qemu.org/2020/09/14/qemu-storage-overview/)
> +in this series introduced QEMU storage concepts. Now we move on to look at
> the
> +two most popular emulated storage controllers for virtualization: virtio-blk
> +and virtio-scsi.
> +
> +This post provides recommendations for configuring virtio-blk and virtio-scsi
> +and how to choose between the two devices. The recommendations provide good
> +performance in a wide range of use cases and are suitable as default settings
> +in tools that use QEMU.
> +
> +## Virtio storage devices
> +### Key points
> +* Prefer virtio storage devices over other emulated storage controllers.
> +* Use the latest virtio drivers.
> +
> +Virtio devices are recommended over other emulated storage controllers as
> they
> +are generally the most performant and fully-featured storage controllers in
> +QEMU.
> +
> +Unlike emulations of hardware storage controllers, virtio-blk and virtio-scsi
> +are specifically designed and optimized for virtualization. The details of
> how
> +they work are published for driver and device implementors in the [VIRTIO
> +specification](https://docs.oasis-open.org/virtio/virtio/v1.1/virtio-v1.1.html).
> +
> +Virtio drivers are available for both Linux and Windows virtual machines.
> +Installing the latest version is recommended for the latest bug fixes and
> +performance enhancements.
> +
> +If virtio drivers are not available, the AHCI (SATA) device is widely
> supported
> +by modern operating systems and can be used as a fallback.
AHCI is only available on q35, so this talk about fallback probably needs
to be worded in a more generic fashion refering to the integrated storage
controller of the machine type.
> +
> +## Comparing virtio-blk and virtio-scsi
> +### Key points
> +* Prefer virtio-scsi for attaching more than 28 disks or for full SCSI
> support.
> +* Prefer virtio-blk in performance-critical use cases.
> +* With virtio-scsi, use scsi-block for SCSI passthrough and otherwise use
> scsi-hd.
> +Two virtio storage controllers are available: virtio-blk and virtio-scsi.
> +
> +### virtio-blk
> +The virtio-blk device presents a block device to the virtual machine. Each
> +virtio-blk device appears as a disk inside the guest. virtio-blk was
> available
> +before virtio-scsi and is the most widely deployed virtio storage controller.
> +
> +The virtio-blk device offers high performance thanks to a thin software stack
> +and is therefore a good choice when performance is a priority.
> +
> +Disks exposed by virtio-blk are typically read-write but can also be
> read-only.
> +They can be used to present read-only ISO images to the guest.
This comment about iSO images feels potentially misleading. You can expose
a volume with any filesystem type to a guest using virtio-blk, but merely
exposing an ISO image is not going to reliably make guest software do the
right thing. Something expecting a CDROM will usaully want a device that
actually looks like a CDROM (ie removable media at very least), not merely
a HDD with ISO formatted content.
I think cloud-init is probably only software example I know where exposing
ISO as HDD is ok-ish.
If anyone wants to expose CDROM to guest, AFAIK, the recommendation has
always been to either use virtio-scsi, or a machine type's built-in
storage controller (IDE, AHCI, or SCSI), never virtio-blk.
> +Applications that send SCSI commands are better served by the virtio-scsi
> +device, which has full SCSI support. SCSI passthrough was removed from the
> +Linux virtio-blk driver in v5.6 in favor of using virtio-scsi.
> +
> +Virtual machines that require access to more than 28 disks can run out of PCI
> +bus slots on i440fx-based machine types since each disk requires its own
> +virtio-blk PCI adapter slot. It is possible to add more virtio-blk devices by
> +extending the virtual machine's PCI busses, but it is simpler to use a single
> +virtio-scsi PCI adapter instead.
You can use mutli-function for virtio-bk, at cost of loosing hotplug, so
saying virtio-blk requires a slot is a little too strict.
The 28 disk image can only be reached if not using other devices, so if
many NICs are present it'll be hit before 28. Maybe talk in slightly
more general terms, such as
"Virtual machines that require access to many disks can hit limits
based on availability of PCI slots, which are under contention
with other devices exposed to the guest such as NICs. For example
a typical i440fx machine type default config allows for about 28
disks. It is possible to use multi-function devices to pack multiple
virtio-blk devies into a single PCI slot at the cost of loosing
hotplug, or add additional PCI busses. Generally though it is
simpler to use a single virtio-scsi PCI adapter instead.
> +
> +### virtio-scsi
> +The virtio-scsi device presents a SCSI Host Bus Adapter to the virtual
> machine.
> +SCSI offers a richer command set than virtio-blk and supports more use cases.
> +
> +Each device supports up to 16,383 LUNs (disks) per target and up to 255
> +targets. This allows a single virtio-scsi device to handle all disks in a
> +virtual machine.
Perhaps add
"Emulated LUNs can be exposed as HDDs or CD-ROMs".
Also while a single virtio-scsi device is sufficient if just considering
max disks, I would think can be desirable to have multiple virtio-scsi
devices if you want to tune placement of iothreads. For example, if guest
has multiple NUMA nodes and you want to expose disks as having affinity to
nodes.
> +
> +SCSI allows access to CD-ROM drives, tapes, and other devices besides disk
> +drives. Physical SCSI devices can be passed through into the virtual machine.
> +
> +Clustering software that uses SCSI Persistent Reservations is supported by
> virtio-scsi, but not by virtio-blk.
> +
> +Performance of virtio-scsi may be lower than virtio-blk due to a thicker
> software stack, but in many use cases, this is not a significant factor. The
> following graph compares 4KB random read performance at various queue depths:
> +
> +
> +
> +### virtio-scsi configuration
> +The following SCSI devices are available with virtio-scsi:
> +
> +|Device|SCSI Passthrough|Performance|
> +|------|----------------|-----------|
> +|scsi-hd|No|Highest|
> +|scsi-block|Yes|Lower|
> +|scsi-generic|Yes|Lowest|
> +
> +The scsi-hd device is suitable for disk image files and host block devices
> +when SCSI passthrough is not required.
> +
> +The scsi-block device offers SCSI passthrough and is preferred over
> +scsi-generic due to higher performance.
> +
> +The following graph compares the sequential I/O performance of these devices
> +using virtio-scsi with an iothread:
> +
> +
> +
> +## Conclusion
> +The virtio-blk and virtio-scsi offer a choice between a single block device
> and
> +a full-fledged SCSI Host Bus Adapter. Virtualized guests typically use one or
> +both of them depending on functional and performance requirements. This post
> +compared the two and offered recommendations on how to choose between them.
> +
> +The next post in this series will discuss the iothreads feature that both
> +virtio-blk and virtio-scsi support for increased performance.
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|