Re: [PATCH] net: add initial support for AF_XDP network backend

[Stefan Hajnoczi]

On Wed, 28 Jun 2023 at 09:59, Jason Wang <jasowang@redhat.com> wrote:
>
> On Wed, Jun 28, 2023 at 3:46 PM Stefan Hajnoczi <stefanha@gmail.com> wrote:
> >
> > On Wed, 28 Jun 2023 at 05:28, Jason Wang <jasowang@redhat.com> wrote:
> > >
> > > On Wed, Jun 28, 2023 at 6:45 AM Ilya Maximets <i.maximets@ovn.org> wrote:
> > > >
> > > > On 6/27/23 04:54, Jason Wang wrote:
> > > > > On Mon, Jun 26, 2023 at 9:17 PM Ilya Maximets <i.maximets@ovn.org> 
> > > > > wrote:
> > > > >>
> > > > >> On 6/26/23 08:32, Jason Wang wrote:
> > > > >>> On Sun, Jun 25, 2023 at 3:06 PM Jason Wang <jasowang@redhat.com> 
> > > > >>> wrote:
> > > > >>>>
> > > > >>>> On Fri, Jun 23, 2023 at 5:58 AM Ilya Maximets <i.maximets@ovn.org> 
> > > > >>>> wrote:
> > > > >> It is noticeably more performant than a tap with vhost=on in terms 
> > > > >> of PPS.
> > > > >> So, that might be one case.  Taking into account that just rcu lock 
> > > > >> and
> > > > >> unlock in virtio-net code takes more time than a packet copy, some 
> > > > >> batching
> > > > >> on QEMU side should improve performance significantly.  And it 
> > > > >> shouldn't be
> > > > >> too hard to implement.
> > > > >>
> > > > >> Performance over virtual interfaces may potentially be improved by 
> > > > >> creating
> > > > >> a kernel thread for async Tx.  Similarly to what io_uring allows.  
> > > > >> Currently
> > > > >> Tx on non-zero-copy interfaces is synchronous, and that doesn't 
> > > > >> allow to
> > > > >> scale well.
> > > > >
> > > > > Interestingly, actually, there are a lot of "duplication" between
> > > > > io_uring and AF_XDP:
> > > > >
> > > > > 1) both have similar memory model (user register)
> > > > > 2) both use ring for communication
> > > > >
> > > > > I wonder if we can let io_uring talks directly to AF_XDP.
> > > >
> > > > Well, if we submit poll() in QEMU main loop via io_uring, then we can
> > > > avoid cost of the synchronous Tx for non-zero-copy modes, i.e. for
> > > > virtual interfaces.  io_uring thread in the kernel will be able to
> > > > perform transmission for us.
> > >
> > > It would be nice if we can use iothread/vhost other than the main loop
> > > even if io_uring can use kthreads. We can avoid the memory translation
> > > cost.
> >
> > The QEMU event loop (AioContext) has io_uring code
> > (utils/fdmon-io_uring.c) but it's disabled at the moment. I'm working
> > on patches to re-enable it and will probably send them in July. The
> > patches also add an API to submit arbitrary io_uring operations so
> > that you can do stuff besides file descriptor monitoring. Both the
> > main loop and IOThreads will be able to use io_uring on Linux hosts.
>
> Just to make sure I understand. If we still need a copy from guest to
> io_uring buffer, we still need to go via memory API for GPA which
> seems expensive.
>
> Vhost seems to be a shortcut for this.

I'm not sure how exactly you're thinking of using io_uring.

Simply using io_uring for the event loop (file descriptor monitoring)
doesn't involve an extra buffer, but the packet payload still needs to
reside in AF_XDP umem, so there is a copy between guest memory and
umem. If umem encompasses guest memory, it may be possible to avoid
copying the packet payload.

Stefan