Re: [PATCH v4 1/1] os-posix: asynchronous teardown for shutdown on Linux

[Daniel P . Berrangé]

On Tue, Oct 25, 2022 at 02:42:11PM +0200, Paolo Bonzini wrote:
> On 8/12/22 15:34, Claudio Imbrenda wrote:
> > This patch adds support for asynchronously tearing down a VM on Linux.
> > 
> > When qemu terminates, either naturally or because of a fatal signal,
> > the VM is torn down. If the VM is huge, it can take a considerable
> > amount of time for it to be cleaned up. In case of a protected VM, it
> > might take even longer than a non-protected VM (this is the case on
> > s390x, for example).
> > 
> > Some users might want to shut down a VM and restart it immediately,
> > without having to wait. This is especially true if management
> > infrastructure like libvirt is used.
> > 
> > This patch implements a simple trick on Linux to allow qemu to return
> > immediately, with the teardown of the VM being performed
> > asynchronously.
> > 
> > If the new commandline option -async-teardown is used, a new process is
> > spawned from qemu at startup, using the clone syscall, in such way that
> > it will share its address space with qemu.The new process will have the
> > name "cleanup/<QEMU_PID>". It will wait until qemu terminates
> > completely, and then it will exit itself.
> > 
> > This allows qemu to terminate quickly, without having to wait for the
> > whole address space to be torn down. The cleanup process will exit
> > after qemu, so it will be the last user of the address space, and
> > therefore it will take care of the actual teardown. The cleanup
> > process will share the same cgroups as qemu, so both memory usage and
> > cpu time will be accounted properly.
> > 
> > If possible, close_range will be used in the cleanup process to close
> > all open file descriptors. If it is not available or if it fails, /proc
> > will be used to determine which file descriptors to close.
> > 
> > If the cleanup process is forcefully killed with SIGKILL before the
> > main qemu process has terminated completely, the mechanism is defeated
> > and the teardown will not be asynchronous.
> > 
> > This feature can already be used with libvirt by adding the following
> > to the XML domain definition to pass the parameter to qemu directly:
> > 
> >    <commandline xmlns="http://libvirt.org/schemas/domain/qemu/1.0";>
> >    <arg value='-async-teardown'/>
> >    </commandline>
> > 
> > Signed-off-by: Claudio Imbrenda<imbrenda@linux.ibm.com>
> > Reviewed-by: Murilo Opsfelder Araujo<muriloo@linux.ibm.com>
> > Tested-by: Murilo Opsfelder Araujo<muriloo@linux.ibm.com>
> > ---
> 
> Nice trick indeed!
> 
> The only question I have is whether it would make sense to do this in
> Libvirt instead.

Hmm, interesting idea, that hadn't occurred to me.

Current flow is

  libvirtd -> fork -> fork -> execve(qemu)
                                 \-> clone(async handler)

Due to CLONE_VM, both main QEMU PID and async handler PID are sharing
the memory mappings of code being executed. Everything is fine.


If doing it in libvirt the flow would be

  libvirtd -> fork -> fork -> execve(qemu)
                        \-> clone(async handler)

In the latter case the async handler would be running code from
the libvirt binary. When the parent calls execve that will load
the code from QEMU, and because of CLONE_VM, this affect the
memory mappings in the async handler proces too. I think that
will end up quite explody, no ?

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|