The first vDPA ioctls have been added to the vhost-user protocol and I
wonder if it's time to fully change the vhost-user protocol's focus to
providing a full VIRTIO device model like vDPA does.
Initially vhost-user was just used for vhost-net. As a result it didn't
need the full VIRTIO device model including the configuration space and
device status register.
Over the years device-specific messages were added to extend vhost-user
to cover more of the VIRTIO device model. vhost-user-blk needed
configuration space support, for example.
The problem for VMMs and device backend implementors is that the
protocol is currently positioned halfway between the original vhost-net
approach and the full VIRTIO device model. Even if a VMM implements
VHOST_USER_GET_CONFIG, it can only expect it to work with
vhost-user-blk, not vhost-user-net. Similarly, a vhost-user-net device
backend cannot implement VHOST_USER_GET_CONFIG and expect all VMMs to
allow it to participate in configuration space emulation because
existing VMMs won't send that message.
The current approach where each device type uses a undocumented subset
of vhost-user messages is really messy. VMM and device backend
implementors have to look at existing implementations to know what is
expected for a given device type. It would be nice to switch to the
VIRTIO device model so that the VIRTIO specification can be used as the
reference for how device types work.
Now that vDPA is here and extends the kernel vhost ioctls with a full
VIRTIO device model, it might be a good time to revise the vhost-user
protocol.
A vdpa-user protocol (or vhost-user 2.0) would replace the current mess
with a full VIRTIO device model. Both VMMs and device backends would
require changes to support this, but it would be a cleaner path forward
for the vhost-user protocol.
One way of doing this would be a new VHOST_USER_PROTOCOL_F_VDPA feature
bit that indicates all the currently existing Linux vDPA ioctl messages
are available. Legacy vhost-user messages with overlapping functionality
must not be used when this bit is set. Most importantly, device backends
need to implement the full VIRTIO device model, regardless of device
type (net, blk, scsi, etc).
The device type most affected by this change would be virtio-net. The
other device types are already closer to the full VIRTIO device model.
I wanted to share this idea in case someone is currently trying to
figure out how to add more VIRTIO device model functionality to
vhost-user.