Re: [PATCH] qmp: Stabilize preconfig

[Markus Armbruster]

Michal Privoznik <mprivozn@redhat.com> writes:

> The -preconfig option and exit-preconfig command are around for
> quite some time now. However, they are still marked as unstable.
> This is suboptimal because it may block some upper layer in
> consuming it. In this specific case - Libvirt avoids using
> experimental features.
>
> Signed-off-by: Michal Privoznik <mprivozn@redhat.com>

If I remember correctly, the motivation for -preconfig was NUMA
configuration via QMP.  More uses may have appeared since.

Back then, I questioned the need for yet another option and yet another
state: why not -S?

The answer boiled down to

0. Yes, having just one would be a simpler and cleaner interface, but

1. the godawful mess QEMU startup has become makes -S unsuitable for
   some things we want to do, so we need -preconfig,

2. which is in turn unsuitable for other things we want to do, so we
   still need -S".

3. Cleaning up the mess to the point where "simpler and cleaner" becomes
   viable again is not in the cards right now.

Details in the sub-thread
    Message-ID: <87zi21apkh.fsf@dusky.pond.sub.org>
    https://lists.gnu.org/archive/html/qemu-devel/2018-04/msg02447.html

My conclusion was

    My gut's reaction to preconfig is as nauseous as ever.  It adds
    complexity we wouldn't want or need with sanely structured program
    startup.  It might still be the best we can do, and something we need to
    do.  You have to take on technical debt sometimes.  To know for sure
    this is the case here, we'd have to explore alternatives more seriously.
    Like repaying some of the existing technical debt around there.  Sadly,
    I lack the time to take on such a project.

    Since I lack the money to put where my mouth is, I'm going to shut up
    and get out of your way.

    Message-ID: <87tvqx12no.fsf@dusky.pond.sub.org>
    https://lists.gnu.org/archive/html/qemu-devel/2018-05/msg05584.html

However, we made it experimental:

commit 361ac948a5c960ce7a093cec1744bff0d5af3dec
Author: Markus Armbruster <armbru@redhat.com>
Date:   Thu Jul 5 11:14:02 2018 +0200

    cli qmp: Mark --preconfig, exit-preconfig experimental
    
    Committing to the current --preconfig / exit-preconfig interface
    before it has seen any use is premature.  Mark both as experimental,
    the former in documentation, the latter by renaming it to
    x-exit-preconfig.
    
    See the previous commit for more detailed rationale.
    
    Signed-off-by: Markus Armbruster <armbru@redhat.com>
    Message-Id: <20180705091402.26244-3-armbru@redhat.com>
    Reviewed-by: Eric Blake <eblake@redhat.com>
    Acked-by: Eduardo Habkost <ehabkost@redhat.com>
    Acked-by: Igor Mammedov <imammedo@redhat.com>
    [Straightforward conflict with commit 514337c142f resolved]

commit 1f214ee1b83afd10fd5e1b63f4ecc03f9a8115c4
Author: Markus Armbruster <armbru@redhat.com>
Date:   Thu Jul 5 11:14:01 2018 +0200

    qapi: Do not expose "allow-preconfig" in query-qmp-schema
    
    According to commit 047f7038f58, option --preconfig
    
        [...] allows pausing QEMU in the new RUN_STATE_PRECONFIG state,
        allowing the configuration of QEMU from QMP before the machine
        jumps into board initialization code of machine_run_board_init()
    
        The intent is to allow management to query machine state and
        additionally configure it using previous query results within one
        QEMU instance (i.e. eliminate the need to start QEMU twice, 1st to
        query board specific parameters and 2nd for actual VM start using
        query results for additional parameters).
    
    The implementation is a bit of a hack: it splices in an additional
    main loop before machine creation, in special runstate preconfig.  New
    command exit-preconfig exits that main loop.  QEMU continues
    initializing, creates the machine, and runs the good old main loop.
    The replacement of the main loop is transparent to monitors.
    
    Sadly, some commands expect initialization to be complete.  Running
    them in --preconfig's main loop violates their preconditions.  Since
    we don't really know which commands are safe, we use a whitelist.
    This drags the concept of run state into the QMP core.
    
    The whitelist is done as a command flag in the QAPI schema (commit
    d6fe3d02e9a).  Drags the concept of run state further into the QAPI
    language.
    
    The command flag is exposed in query-qmp-schema (also commit
    d6fe3d02e9a).  This makes it ABI.
    
    I consider the whole thing an offensively ugly hack, but sometimes an
    ugly hack is the best we can do to solve a problem people have.
    
    The need described by the commit message quote above is genuine.  The
    proper solution would be a main loop that permits complete
    configuration via QMP.  This is out of reach, thus the hack.
    
    However, even though the need is genuine, it isn't urgent: libvirt is
    not going to use this anytime soon.  Baking a hack into ABI before it
    has any users is a bad idea.
    
    This commit reverts the parts of commit d6fe3d02e9a that affect ABI
    via query-qmp-schema.  The commit did the following:
    
    (1) Add command flag 'allow-preconfig' to the QAPI schema language
    
    (2) Pass it to code generators
    
    (3) Have the commands.py code generator pass it to the command
        registry (so commit 047f7038f58 can use it as whitelist)
    
    (4) Add 'allow-preconfig' to SchemaInfoCommand (neglecting to update
        qapi-code-gen.txt section "Client JSON Protocol introspection")
    
    (5) Set 'allow-preconfig': true for commands qmp_capabilities,
        query-commands, query-command-line-options, query-status
    
    Revert exactly (4), plus a bit of documentation added to
    qemu-tech.info in commit 047f7038f58.
    
    Shrinks query-qmp-schema's output from 126.5KiB to 121.8KiB for me.
    
    Signed-off-by: Markus Armbruster <armbru@redhat.com>
    Message-Id: <20180705091402.26244-2-armbru@redhat.com>
    Reviewed-by: Eric Blake <eblake@redhat.com>
    Acked-by: Eduardo Habkost <ehabkost@redhat.com>
    Acked-by: Igor Mammedov <imammedo@redhat.com>
    [Straightforward conflict with commit d626b6c1ae7 resolved]

Before we make it a stable interface, we should ponder:

* Is cleaning up the mess to the point where "simpler and cleaner"
  becomes viable again still impractical?

* If yes, what are the chances of it becoming practical?