Re: [PATCH V1 15/32] vl: QEMU_START_FREEZE env var

[Dr. David Alan Gilbert]

* Steven Sistare (steven.sistare@oracle.com) wrote:
> On 9/11/2020 2:49 PM, Dr. David Alan Gilbert wrote:
> > * Steve Sistare (steven.sistare@oracle.com) wrote:
> >> For qemu upgrade and restart, we will re-exec() qemu with the same argv.
> >> However, qemu must start in a paused state and wait for the cprload 
> >> command,
> >> and the original argv might not contain the -S option.  To avoid modifying
> >> argv, provide the QEMU_START_FREEZE environment variable.  If
> >> QEMU_START_FREEZE is set, then set autostart=0, like the -S option.
> >>
> >> Signed-off-by: Steve Sistare <steven.sistare@oracle.com>
> > 
> > What's wrong with modifying the argv?
> > 
> > Note, also the trick -incoming defer uses;  the whole point here is that
> > we start qemu with   -incoming defer     and then we can issue commands
> > to modify the QEMU configuration before we actually reload state.
> > 
> > Note, even without CPR there might be reasons that you need to modify
> > the argv; for example, imagine that since it was originally booted
> > someone had hotplug added an extra CPU or RAM or a disk; the new QEMU
> > must be started in a state that reflects the state in which the VM was
> > at the point when it was saved, not the point at which it was started
> > long ago.
> 
> The code is simpler if we do not need to parse and massage the argv, and that 
> is 
> sufficient for many use cases.  QEMU_START_FREEZE adds only a few lines of 
> code, and 
> it's nice to have that choice.
> 
> For hot plug, we rely on the management layer to know what devices were 
> plugged
> after the initial startup, and re-plug them after restart.  cprsave restarts 
> qemu,
> which creates command-line devices.  At this point the manager would send the 
> hotplug 
> commands (just like -incoming defer), then send cprload. 
> 
> Having said that, if the management layer sometimes performs live migration, 
> and sometimes
> performs cpr restart, then we need to strip out any -incoming args from argv 
> before restart.
> This can be done in the vendor-specific qemu-exec helper (patch 20).

My problem is I can see a whole bunch of places that reusing the
original argv breaks, so I don't think this is a useful general
solution:

   a) The -incoming example
   b) The management app has to reply the hotplug sequence
   c) ...even if it did there's no guarantee that the original
pre-hotplug commandline works:
      i) e.g. an original block device file was deleted
     ii) One of the endpoints for a network device is gone.

  Any part of (c) could cause the exec'd qemu to fail before
it gets as far as allowing you to issue the hotplug commands.
It's also plain dangerous, since the exec'd qemu shouldn't be accessing
a  file or device that has been hot-unplugged and might now be part of
a different VM.

So I think you really should pass another command line option here
rather than setting an environment variable; and then I think you should
consider two separate things:

  a) You could easily strip out options of the form --cpr-freeze
  b) Consider something more general; e.g. allow the management layer to
specify a new set of argv to be used by the exec.

Dave

> - Steve
> 
> >> ---
> >>  softmmu/vl.c | 5 +++++
> >>  1 file changed, 5 insertions(+)
> >>
> >> diff --git a/softmmu/vl.c b/softmmu/vl.c
> >> index 951994f..7016e39 100644
> >> --- a/softmmu/vl.c
> >> +++ b/softmmu/vl.c
> >> @@ -4501,6 +4501,11 @@ void qemu_init(int argc, char **argv, char **envp)
> >>          exit(0);
> >>      }
> >>  
> >> +    if (getenv("QEMU_START_FREEZE")) {
> >> +        unsetenv("QEMU_START_FREEZE");
> >> +        autostart = 0;
> >> +    }
> >> +
> >>      if (incoming) {
> >>          Error *local_err = NULL;
> >>          qemu_start_incoming_migration(incoming, &local_err);
> >> -- 
> >> 1.8.3.1
> >>
> 
-- 
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK