Re: Making QEMU easier for management tools and applications

[Daniel P . Berrangé]

On Sat, Dec 21, 2019 at 10:02:23AM +0100, Markus Armbruster wrote:
> Stefan Hajnoczi <address@hidden> writes:
> 
> > Hi,
> > QEMU presents a command-line interface and QMP monitor for
> > applications to interact with.  Applications actually need API
> > bindings in their programming language.  Bindings avoid reimplementing
> > code to spawn a QEMU process and interact with QMP.  QEMU is kind of
> > lazy and de facto relies on libvirt for API bindings.
> >
> > Is it time for better QEMU APIs?

> * scripts/qmp/qmp-shell
> 
>   Half-hearted attempt at a human-friendly wrapper around the JSON
>   syntax.  I have no use for this myself.

I use this fairly often as its a useful debugging / experimentation
/ trouble shooting tool. There's similar ish functionality in
virsh qemu-monitor-command. I think there's scope of a supported
tool here that can talk to libvirt or a UNIX socket for doing
QMP commands, with a friendlier syntax & pretty printing. 

> > 4. Go and Rust bindings would also be useful.  There is
> > https://github.com/intel/govmm but I think it makes sense to keep it
> > in qemu.git and provide an interface similar to our Python modules.
> 
> Mapping QAPI/QMP commands and events to function signatures isn't hard
> (the QAPI code generator does).  Two problems (at least):
> 
> 1. Leads to some pretty ridiculous functions.  Here's one:
> 
>     void qmp_blockdev_mirror(bool has_job_id, const char *job_id,
>                              const char *device,
>                              const char *target,
>                              bool has_replaces, const char *replaces,
>                              MirrorSyncMode sync,
>                              bool has_speed, int64_t speed,
>                              bool has_granularity, uint32_t granularity,
>                              bool has_buf_size, int64_t buf_size,
>                              bool has_on_source_error,
>                              BlockdevOnError on_source_error,
>                              bool has_on_target_error, BlockdevOnError 
> on_target_error,
>                              bool has_filter_node_name, const char 
> *filter_node_name,
>                              bool has_copy_mode, MirrorCopyMode copy_mode, 
>                              bool has_auto_finalize, bool auto_finalize,
>                              bool has_auto_dismiss, bool auto_dismiss,
>                              Error **errp);
> 
>   We commonly use 'boxed': true for such beasts, which results in
>   functions like this one:
> 
>     void qmp_blockdev_add(BlockdevOptions *arg, Error **errp);
> 
> 2. Many schema changes that are nicely backward compatible in QMP are
>    anything but in such an "obvious" C API.  Adding optional arguments,
>    for instance, or changing integer type width.  The former is less of
>    an issue with 'boxed': true.
> 
> Perhaps less of an issue with dynamic languages.
> 
> I figure a static language would need much more expressive oomph than C
> to be a good target.  No idea how well Go or Rust bindings can work.

In libvirt we've got this kind of problem already and have two different
approaches to tackling it.

For the really complex cases with lots of conceptual nesting, we would
end up using XML.

For the simpler case where its just a set of flat parameters, we have
something we call "virTypedParameters" which is effectively a poor man's
hash table where the keys are parameter names, and the values are a
union accepting different types.

In the Go APIs for libvirt though we end up mapping both those approaches
into Go structs. The key reason why we are able todo this in Go and not
C, is that we don't promise ABI compat in Go bindings. We'll extend the
structs at will, and some times a struct field gets turned into a nested
struct which is an API break too. With Go apps typically bundle all their
3rd party deps & often lock themselves to a specific release tag, so they
don't need ABI compat in the same way apps want at C level.

> > 6. A configuration file format is sorely needed so that guest
> > configuration can be persisted and easily launched.  Today you have to
> > create a shell script that executes a QEMU command-line, but this is
> > suboptimal because sharing executable scripts is dangerous from a
> > security perspective and is hard to parse or modify programmatically.
> 
> No argument.  There is -readconfig, but it falls way short.
> 
> With command line QAPIfication, a real configuration file will be quite
> feasible.
> 
> The main reason for the lack of progress there is our dedication to
> backward compatibility.  A functional replacement of our CLI is a huge
> task already.  Throwing in backward compatibility makes it a daunting
> one.  Not least because nobody fully understands all the quirks.

I think we need to re-think how we're attacking this problem, because
despite heroic efforts in developing QAPI & converting code it use it,
we've still not got the finish line in sight after 10 years.

As I mentioned in my other reply I think we should target the CLI as
something exclusively for humans, with convenient syntax and more
relaxed back compat requirements. ie don't guarantee perfect back
compat for the CLI - just modest effor to not gratuitously break it
without cause. Humans using the CLI can adapt if they hit obscure
quirks.

Our back compat pain with the quirks is because we have mgmt apps using
the CLI which can't tolerate semantic changes without warning. We can't
be more relaxed with CLI back compat without getting mgmt apps away
from using this CLI.  We can get mgmt apps off using the CLI until they
have an alternative they can use.

The hard part is that developing the QAPI alternative is that its impl
might accidentaly cause breakage in existing CLI before mgmt apps can
switch to the QAPI based approach.



I'm wondering if we need to think about a temporary *throwaway* fork of
QEMU. I wasn't involved in the work, but as an outside observer I think
it is interesting to see how the NEMU project fork ultimately led to QEMU
moving forward with a new KConfig approach, and the integration of microvm.
They had the freedom to develop these new approaches with zero regard
to back compat or broader QEMU needs that might otherwise hold back dev.
Once the forked impl was mostly done, "unknown problems" had largely become
"known problems", and the work could be integrated back into QEMU with
much less risk of disruption.


Could such a temporary fork approach help us solve the QAPI-ification
end goal ?  It doesn't have to be a separate project - it could be a
temporary qemu-qapi.git on qemu-project.org or github for experimentation,
but still under umbrealla of QEMU.

It would free us to implement an idealized 100% QAPI-ified config
approach in the shortest possible time, with zero regard to back
compat of the current CLI.

We could develop support for this in libvirt in parallel, letting
us test the combination to gain confidence in the result.

Essentially we'll be aiming to flush out all the "unknown problems"
that hold us back today. As we get confidence we'll be able to merge
it back into QEMU proper with much lower risk of introducing suprising
compat problems.

At some level you can consider each subsys maintainer's branch a fork,
but those branches are really focused on stable code that's ready to
merge. An NEMU like fork is more aggressive than that, as it provided
a focused place for *experimentation* with no regard for immediate
merge to GIT master. The cost of such a fork approach of course is
that of trying to keep up2date with development in QEMU git master.

The benefit of being able to cut the back-compat knot might still
make this a win, allowing completion of QAPI-ification in a faster
timeframe.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|