Re: [Qemu-devel] [RFC 0/1] Add live migration support to the PVRDMA device

[Dr. David Alan Gilbert]

* Marcel Apfelbaum (address@hidden) wrote:
> Hi Dmitry,
> 
> On 6/25/19 11:39 AM, Dmitry Fleytman wrote:
> > 
> > > On 25 Jun 2019, at 11:14, Marcel Apfelbaum <address@hidden> wrote:
> > > 
> > > Hi Sukrit
> > > 
> > > On 6/21/19 5:45 PM, Sukrit Bhatnagar wrote:
> > > > Hi,
> > > [...]
> > > > This RFC is meant to request suggestions on the things which are
> > > > working and for help on the things which are not.
> > > > 
> > > [...]
> > > > What is not working:
> > > > 
> > > [...]
> > > > * It seems that vmxnet3 migration itself is not working properly, at 
> > > > least
> > > >    for me. The pvrdma device depends on it, vmxnet3 is function 0 and 
> > > > pvrdma
> > > >    is function 1. This is happening even for a build of unmodified code 
> > > > from
> > > >    the master branch.
> > > >    After migration, the network connectivity is lost at destination.
> > > >    Things are fine at the source before migration.
> > > >    This is the command I am using at src:
> > > > 
> > > >    sudo /home/skrtbhtngr/qemu/build/x86_64-softmmu/qemu-system-x86_64 \
> > > >      -enable-kvm \
> > > >      -m 2G -smp cpus=2 \
> > > >      -hda /home/skrtbhtngr/fedora.img \
> > > >      -netdev tap,id=hostnet0 \
> > > >      -device vmxnet3,netdev=hostnet0,id=net0,mac=52:54:00:99:ff:bc \
> > > >      -monitor telnet:127.0.0.1:4444,server,nowait \
> > > >      -trace events=/home/skrtbhtngr/trace-events \
> > > >      -vnc 0.0.0.0:0
> > > > 
> > > >    Similar command is used for the dest. Currently, I am trying
> > > >    same-host migration for testing purpose, without the pvrdma device.
> > > >    Two tap interfaces, for src and dest were created successfully at
> > > >    the host. Kernel logs:
> > > >    ...
> > > >    br0: port 2(tap0) entered forwarding state
> > > >    ...
> > > >    br0: port 3(tap1) entered forwarding state
> > > > 
> > > >    tcpdump at the dest reports only outgoing ARP packets, which ask
> > > >    for gateway: "ARP, Request who-has _gateway tell guest1".
> > > > 
> > > >    Tried using user (slirp) as the network backend, but no luck.
> > > >       Also tried git bisect to find the issue using a working commit 
> > > > (given
> > > >    by Marcel), but it turns out that it is very old and I faced build
> > > >    errors one after another.
> > > > 
> > > >    Please note that e1000 live migration is working fine in the same 
> > > > setup.
> > > > 
> > > I tried to git bisect , but I couldn't find a working version of vmxnet 
> > > supporting live migration ....
> > > I tried even a commit from December 2014 and it didn't work.
> > > 
> > > What is strange (to me) is that the networking packets can't be sent from 
> > > the guest (after migration)
> > > even after rebooting the guest.
> > This makes me think that some network offload configuration wasn’t properly 
> > migrated or applied.
> > What network backend are you using?
> 
> Suktrit tried with tap device, I tried with slirp.
> If you can point me to the property that disables all the offloads it will
> really help.
> 
> > Do you see any outgoing packets in the sniffer?
> 
> I didn't use the sniffer, I checked dmesg in guest, there was a line
> complaining that it can't send packets.

What exactly was the error?

I don't know much about vmxnet3;  but if the guest driver is seeing the
problem then I guess that's the best pointer we have.


Dave

> Thanks,
> Marcel
> 
> > > Any help or pointer would be greatly appreciated.
> > > Thanks,
> > > Marcel
> > > 
> > > 
> > > [...]
> 
--
Dr. David Alan Gilbert / address@hidden / Manchester, UK