Re: [PATCH v3] virtio-net: prevent offloads reset on migration

[Michael S. Tsirkin]

On Thu, Oct 10, 2019 at 02:11:15PM +0200, Mikhail Sennikovsky wrote:
> Currently offloads disabled by guest via the 
> VIRTIO_NET_CTRL_GUEST_OFFLOADS_SET
> command are not preserved on VM migration.
> Instead all offloads reported by guest features (via 
> VIRTIO_PCI_GUEST_FEATURES)
> get enabled.
> What happens is: first the VirtIONet::curr_guest_offloads gets restored
> and offloads are getting set correctly:
> 
>  #0  qemu_set_offload (nc=0x555556a11400, csum=1, tso4=0, tso6=0, ecn=0, 
> ufo=0) at net/net.c:474
>  #1  virtio_net_apply_guest_offloads (n=0x555557701ca0) at 
> hw/net/virtio-net.c:720
>  #2  virtio_net_post_load_device (opaque=0x555557701ca0, version_id=11) at 
> hw/net/virtio-net.c:2334
>  #3  vmstate_load_state (f=0x5555569dc010, vmsd=0x555556577c80 
> <vmstate_virtio_net_device>, opaque=0x555557701ca0, version_id=11)
>      at migration/vmstate.c:168
>  #4  virtio_load (vdev=0x555557701ca0, f=0x5555569dc010, version_id=11) at 
> hw/virtio/virtio.c:2197
>  #5  virtio_device_get (f=0x5555569dc010, opaque=0x555557701ca0, size=0, 
> field=0x55555668cd00 <__compound_literal.5>) at hw/virtio/virtio.c:2036
>  #6  vmstate_load_state (f=0x5555569dc010, vmsd=0x555556577ce0 
> <vmstate_virtio_net>, opaque=0x555557701ca0, version_id=11) at 
> migration/vmstate.c:143
>  #7  vmstate_load (f=0x5555569dc010, se=0x5555578189e0) at 
> migration/savevm.c:829
>  #8  qemu_loadvm_section_start_full (f=0x5555569dc010, mis=0x5555569eee20) at 
> migration/savevm.c:2211
>  #9  qemu_loadvm_state_main (f=0x5555569dc010, mis=0x5555569eee20) at 
> migration/savevm.c:2395
>  #10 qemu_loadvm_state (f=0x5555569dc010) at migration/savevm.c:2467
>  #11 process_incoming_migration_co (opaque=0x0) at migration/migration.c:449
> 
> However later on the features are getting restored, and offloads get reset to
> everything supported by features:
> 
>  #0  qemu_set_offload (nc=0x555556a11400, csum=1, tso4=1, tso6=1, ecn=0, 
> ufo=0) at net/net.c:474
>  #1  virtio_net_apply_guest_offloads (n=0x555557701ca0) at 
> hw/net/virtio-net.c:720
>  #2  virtio_net_set_features (vdev=0x555557701ca0, features=5104441767) at 
> hw/net/virtio-net.c:773
>  #3  virtio_set_features_nocheck (vdev=0x555557701ca0, val=5104441767) at 
> hw/virtio/virtio.c:2052
>  #4  virtio_load (vdev=0x555557701ca0, f=0x5555569dc010, version_id=11) at 
> hw/virtio/virtio.c:2220
>  #5  virtio_device_get (f=0x5555569dc010, opaque=0x555557701ca0, size=0, 
> field=0x55555668cd00 <__compound_literal.5>) at hw/virtio/virtio.c:2036
>  #6  vmstate_load_state (f=0x5555569dc010, vmsd=0x555556577ce0 
> <vmstate_virtio_net>, opaque=0x555557701ca0, version_id=11) at 
> migration/vmstate.c:143
>  #7  vmstate_load (f=0x5555569dc010, se=0x5555578189e0) at 
> migration/savevm.c:829
>  #8  qemu_loadvm_section_start_full (f=0x5555569dc010, mis=0x5555569eee20) at 
> migration/savevm.c:2211
>  #9  qemu_loadvm_state_main (f=0x5555569dc010, mis=0x5555569eee20) at 
> migration/savevm.c:2395
>  #10 qemu_loadvm_state (f=0x5555569dc010) at migration/savevm.c:2467
>  #11 process_incoming_migration_co (opaque=0x0) at migration/migration.c:449
> 
> Fix this by making the virtio_net_set_features not reset
> the curr_guest_offloads in case the VM runstate is RUN_STATE_INMIGRATE
> 
> Signed-off-by: Mikhail Sennikovsky <address@hidden>

I thought more about it and I really think the problem here is
different: we really should first set features and then apply offloads.

It is simply dumb that we are calling device specific
code before initializing generic virtio.

The logical order of initialization should be:

init base class
init derived class


instead we have it all spread out :(



What to do about it?
Could we maybe teach vmstate to record post load hooks
it needs to run *somewhere*?
Then we could load device specific state and record post load
hooks, then load generic state, run generic post load,
then run device specific post load hooks.

Dave, what do you think?


> ---
>  hw/net/virtio-net.c | 9 ++++++++-
>  1 file changed, 8 insertions(+), 1 deletion(-)
> 
> diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
> index b9e1cd7..49be172 100644
> --- a/hw/net/virtio-net.c
> +++ b/hw/net/virtio-net.c
> @@ -767,7 +767,14 @@ static void virtio_net_set_features(VirtIODevice *vdev, 
> uint64_t features)
>      n->rsc6_enabled = virtio_has_feature(features, VIRTIO_NET_F_RSC_EXT) &&
>          virtio_has_feature(features, VIRTIO_NET_F_GUEST_TSO6);
>  
> -    if (n->has_vnet_hdr) {
> +    /*
> +     * In case of RUN_STATE_INMIGRATE the virtio_net_set_features
> +     * is called as part of VM state restore process.
> +     * At this stage we do not want the curr_guest_offloads to be reset,
> +     * i.e. want to preserve them in the same state as was set
> +     * by the guest on the source machine.
> +     */
> +    if (n->has_vnet_hdr && !runstate_check(RUN_STATE_INMIGRATE)) {
>          n->curr_guest_offloads =
>              virtio_net_guest_offloads_by_features(features);
>          virtio_net_apply_guest_offloads(n);
> -- 
> 2.7.4