Am 29.09.2021 um 14:18 hat Markus Armbruster geschrieben:
> > I'm proposing a one-off here just for some initial discussion and planning.
> > (We didn't have a QAPI BoF at KVM Forum and I'd like something similar
> > now.) I don't expect we'll really "finish" plotting out a roadmap in a
> > single 60 minute meeting, but it will probably give us good, targeted
> > discussions on the ML to follow. If more meetings are desired to discuss
> > specific topics, I'm not against scheduling them as-needed.
>
> Let's have one now, and then more as needed.
>
> Jumping into the meeting unprepared, then spending half of the alloted
> time listening to people explaining where they want to take QAPI seems
> like a bad use of our time. Better: write it up and post it, so we can
> all read before we meet. Do not reply, except perhaps to ask for
> clarification. Discuss in the meeting.
>
> I can see such posts from Marc-Andre, Kevin, and Daniel. Some of them
> could use a bit more detail, perhaps.
To be a bit more specific and give more details about what I'm currently
working on:
* The larger theme is command line QAPIfication, with the eventual goals
of having introspection and a config file.
* Support for member aliases: This provides a way to bridge much of the
gap between command line and QMP syntax. Markus tells me that it's
unclear yet whether we want this in both QMP and CLI (unlikely for
now), just CLI or not at all.
First posted in November 2020, latest version is v4:
https://patchew.org/QEMU/20210917161320.201086-1-kwolf@redhat.com/
* Does anyone apart from myself have additional uses for aliases that
should be considered in the evaluation of the feature?
* -chardev: I have patches that QAPIfy the option based on aliases,
getting rid of the old handwritten parser that is inconsistent with
QMP in non-obvious ways and replacing it with translation to QMP
(both using aliases and a little C code) that makes the differences
obvious.
First posted in November 2020, more details in the cover letter:
https://patchew.org/QEMU/20201112175905.404472-1-kwolf@redhat.com/
Later versions (not yet posted as a series because I'm waiting for
aliases) also make -chardev accept JSON syntax, which is what
libvirt really wants to use.
* Came up during the discussion of the patches:
Maybe we should actually start defining some command line options in
the schema now, even if many options will still have to defer to the
old parsing code. But it would be a start, could set an example for
any new options and provides a clear path to incrementally convert
everything. If we don't even start, we're sure to never finish it.
* -object: First part is done, we have a full QAPI schema of all
user-creatable objects. object-add and --object in tools go through
pure QAPI paths. System emulator -object is still based on QemuOpts
because of compatibility concerns with list hacks (affects one single
property of one object), but JSON syntax has been added that goes
straight to QAPI.
Next part is actual integration of QOM with QAPI so that the
configuration isn't duplicated between QOM code and the schema, but
user-creatable QOM classes are defined and QOM boilerplate is
generated from the schema. I've started looking into this, but still
in an early stage.
* -device/device-add: This is in a terrible state all around. Even QMP
device-add is based on QemuOpts, and libvirt makes use of this by
passes everything as strings instead of the proper types.
Peter Krempa is working on using the proper types in libvirt.
I sent patches to at least add a JSON variant for the command line
option -device which is a non-QemuOpts code path. It also tried to fix
device-add to be type safe, but until libvirt is fixed, we must stay
bug compatible. So for this and other reasons this is not ready yet
and needs a v2.
https://patchew.org/QEMU/20210924090427.9218-1-kwolf@redhat.com/
Full QAPIfication with a schema and introspection will still take a
while. The QOM/QAPI integration work mentioned above is a dependency
there. A schema like for object-add, with a big union of all possible
devices, just doesn't scale.
Having JSON CLI and a non-QemuOpts QMP command allow us to make the
traditional syntax unstable (HMP-like support) and should be very
helpful when we eventually do get to do proper QAPIfication.
* Should we have a general rule that for command line options that
accept both JSON and a human format, only JSON is considered a
stable API and the human format works more like HMP?
Also one non-technical topic that may or may not be worth discussing:
* Patches to QAPI tend to take a long time to be merged (or rejected).
This is understandable as a lot of things are going on and QAPI is an
external interface with compatibility promises, so we need to be
careful. Can we still still do anything to improve the situation? For
example, can the load be spread across more people?
Kevin
I'll drop by and leave an extremely crude summary here; I had a monster headache all day yesterday that has still not left me, my apologies for this being a bit close to the wire:
My main motivators for QAPI are:
(1) A QEMU config file that is ultimately backed by the same schema as QMP.
Ideally, this config file would have intellisense syntax support in e.g. pycharm/vscode. There is a large repository of "config file schemas" online used for auto-complete and intellisense for various YAML/JSON configuration files. I started researching this option around last year, for KVM Forum. I began by giving a presentation on the state of the CLI as it existed at the time, and submitted a demo that involved a very crudely hacked up "x-configure" QMP command that re-routed to the CLI parser. It worked only because I was fairly aggressive about removing a good number of CLI arguments.
I consider this important for ease-of-use and discoverability reasons.
(2) The unification of QAPI and QDEV
There are numerous spots in the API where we stub out to an 'Any' type, because the actual type information is in qdev or open-coded in various device functions. In order to support the above goal, I see the unification of the two dynamic systems as necessary for producing significantly improved static schema that will facilitate users in discovering and using QEMU's features.
This goal serves primarily as reinforcement for the first.
(3) The generation of an improved QAPI index in the Sphinx ReST docs.
At present, we do not have cross-references for QMP commands, events, enums, structs, arguments, etc. I wrote a prototype "QAPI Domain" plugin for Sphinx earlier this year, but there is work remaining to integrate it to the current QAPIDoc class in the QAPI generator. In essence, I have a "new" syntax that works to accomplish this goal, but I don't have a translator from the old to new syntax yet. I also do not have much of the error checking that we'd want to enforce layout, style, etc.
This will help further bolster the discoverability and ease of use for our API. It will allow me to write ReST documents that natively cross-reference the exact commands and structures I am talking about elsewhere in the docs. It will allow me to reference QMP command documentation from Python docstrings, for instance, which would be very nice for a Python SDK ...
(4) Libraries and SDKs for Python, Golang, and rust.
I've been working on the Python angle here, because we already have all of the pieces for a python library. I want to package them and release them on PyPI as developer tooling and "advanced user" tooling. Not something meant to replace libvirt, but instead just well-supported, minimalist FOSS tooling that allows you to drive and experiment with cutting-edge features of QEMU. The intended audience is power users and developers of other software that consumes QEMU. I have noticed many projects that wind up relying on QEMU tend to reimplement the same code (sometimes poorly) over and over again -- I want to provide a bit more first class support to our API.
To this end, I've spent the last year cleaning up our Python code quite a bit, including adding documentation and static typing to the QAPI generator itself, in the hopes that this will facilitate adding new languages and generators. I've also recently checked in a new version of the python QMP library; if it tests well this release cycle I am hoping to release it on PyPI this year.