Re: [Qemu-devel] [RFC PATCH 0/6] Enhancing Qemu MMIO emulation with scripting interface

[Balamuruhan S]

On Wed, Aug 07, 2019 at 10:15:48AM +0200, Cédric Le Goater wrote:
> On 07/08/2019 09:14, Balamuruhan S wrote:
> > Hi All,
> > 
> > This is a proposal to extend mmio callbacks in Qemu with scripting interface
> > that is prototyped with python in this implementation. It gives ability to
> > feed runtime data through callbacks without recompiling Qemu in generic way.
> > This patchset adds library that provides APIs for Qemu to talk with python
> > scripts placed in path -module-path and how existing xscom can be extended
> > with python interface infrastructure.
> > 
> > We have also added an hacky emulation for memory region (OCC common area 
> > and HOMER)
> > which is shared between core and un-core engine (ideally this should be via
> > sram device) to showcase the effectiveness of having the scripting interface
> > (uncore engine taken for discussion here is powerpc specificed called OCC).
> 
> We should try to merge this part first. It is useful as it is after some
> cleanups.

okay :+1:

> 
> > Having scripting interface helps to emulate/test different uncore-core
> > interactions including uncore engine failure or hang. It also helps in 
> > feeding
> > randomized data at byte level access. This patchset is primarily to extend 
> > mmio
> > callbacks with scripting interface and to demonstrate effectiveness it.
> 
> It is already possible to feed device models with external data using QMP or
> external agents using a chardev backend transport. What are the benefits
> of using the embedded python approach ?  
> 
> > Some changes are required in PowerPC skiboot tree to test these changes 
> > since
> > the memory region is disabled currently for Qemu emulated PowerNV host,
> > https://github.com/balamuruhans/skiboot/commit/a655514d2a730e0372a2faee277d1cf01f71a524
> 
> You should send that patch.

okay, I will send it.

Thank you Cedric for your review and suggestions.

-- Bala
> 
> Thanks,
> 
> C. 
> 
> > Qemu commandline used to test,
> > 
> > ```
> > # qemu/ppc64-softmmu/qemu-system-ppc64 \
> > -M powernv \
> > -cpu POWER9 \
> > -m 16G \
> > -kernel vmlinux \
> > -initrd debug_homer.cpio \
> > -nographic -bios skiboot/skiboot.lid \
> > -module-path 
> > /home/bala/homer/python-modules/,xscom_module=homer,xscom_read=xscom_read,xscom_write=xscom_write,homer_module=homer,homer=homer_read,occ_module=homer,occ=occ_read
> > ```
> > 
> > Script used to feed data can be something like,
> > https://github.com/balamuruhans/python-modules/blob/master/script.py
> > 
> > It could uncover couple of firmware bugs,
> > https://github.com/balamuruhans/skiboot/commit/fd3d93d92ec66a7494346d6d24ced7b48264c9a0
> > https://github.com/balamuruhans/skiboot/commit/165b3829a93bc177c18133945a8cca3a2d701173
> > 
> > Code changes:
> > Patch 1: adds library to provide python interface APIs
> > Patch 2: extend existing xscom to adopt this python interface
> > Patch 3 - 6: emulate uncore/core shared memory region with mmio callbacks 
> > and
> > add support with this infrastructure.
> > 
> > I request for comments, suggestions, ideas on getting a scripting interface
> > like python added in qemu.
> > 
> > Balamuruhan S (6):
> >   utils/python_api: add scripting interface for Qemu with python lib
> >   hw/ppc/pnv_xscom: extend xscom to use python interface
> >   hw/ppc/pnv_homer: add homer/occ common area emulation for PowerNV
> >   hw/ppc/pnv: initialize and realize homer/occ common area
> >   hw/ppc/pnv_xscom: retrieve homer/occ base address from PBA BARs
> >   hw/ppc/pnv_homer: add python interface support for homer/occ common
> >     area
> > 
> >  configure                   |  10 +++
> >  hw/ppc/Makefile.objs        |   2 +-
> >  hw/ppc/pnv.c                |  49 ++++++++++-
> >  hw/ppc/pnv_homer.c          | 205 
> > ++++++++++++++++++++++++++++++++++++++++++++
> >  hw/ppc/pnv_xscom.c          |  59 +++++++++++--
> >  include/hw/ppc/pnv.h        |  15 ++++
> >  include/hw/ppc/pnv_homer.h  |  41 +++++++++
> >  include/sysemu/python_api.h |  30 +++++++
> >  include/sysemu/sysemu.h     |   8 ++
> >  qemu-options.hx             |  14 +++
> >  util/Makefile.objs          |   1 +
> >  util/python_api.c           | 100 +++++++++++++++++++++
> >  vl.c                        |  66 ++++++++++++++
> >  13 files changed, 588 insertions(+), 12 deletions(-)
> >  create mode 100644 hw/ppc/pnv_homer.c
> >  create mode 100644 include/hw/ppc/pnv_homer.h
> >  create mode 100644 include/sysemu/python_api.h
> >  create mode 100644 util/python_api.c
> > 
>