Re: [Qemu-devel] [RFC PATCH 0/6] Enhancing Qemu MMIO emulation with scripting interface

[Cédric Le Goater]

On 07/08/2019 09:14, Balamuruhan S wrote:
> Hi All,
> 
> This is a proposal to extend mmio callbacks in Qemu with scripting interface
> that is prototyped with python in this implementation. It gives ability to
> feed runtime data through callbacks without recompiling Qemu in generic way.
> This patchset adds library that provides APIs for Qemu to talk with python
> scripts placed in path -module-path and how existing xscom can be extended
> with python interface infrastructure.
> 
> We have also added an hacky emulation for memory region (OCC common area and 
> HOMER)
> which is shared between core and un-core engine (ideally this should be via
> sram device) to showcase the effectiveness of having the scripting interface
> (uncore engine taken for discussion here is powerpc specificed called OCC).

We should try to merge this part first. It is useful as it is after some
cleanups.

> Having scripting interface helps to emulate/test different uncore-core
> interactions including uncore engine failure or hang. It also helps in feeding
> randomized data at byte level access. This patchset is primarily to extend 
> mmio
> callbacks with scripting interface and to demonstrate effectiveness it.

It is already possible to feed device models with external data using QMP or
external agents using a chardev backend transport. What are the benefits
of using the embedded python approach ?  

> Some changes are required in PowerPC skiboot tree to test these changes since
> the memory region is disabled currently for Qemu emulated PowerNV host,
> https://github.com/balamuruhans/skiboot/commit/a655514d2a730e0372a2faee277d1cf01f71a524

You should send that patch.

Thanks,

C. 

> Qemu commandline used to test,
> 
> ```
> # qemu/ppc64-softmmu/qemu-system-ppc64 \
> -M powernv \
> -cpu POWER9 \
> -m 16G \
> -kernel vmlinux \
> -initrd debug_homer.cpio \
> -nographic -bios skiboot/skiboot.lid \
> -module-path 
> /home/bala/homer/python-modules/,xscom_module=homer,xscom_read=xscom_read,xscom_write=xscom_write,homer_module=homer,homer=homer_read,occ_module=homer,occ=occ_read
> ```
> 
> Script used to feed data can be something like,
> https://github.com/balamuruhans/python-modules/blob/master/script.py
> 
> It could uncover couple of firmware bugs,
> https://github.com/balamuruhans/skiboot/commit/fd3d93d92ec66a7494346d6d24ced7b48264c9a0
> https://github.com/balamuruhans/skiboot/commit/165b3829a93bc177c18133945a8cca3a2d701173
> 
> Code changes:
> Patch 1: adds library to provide python interface APIs
> Patch 2: extend existing xscom to adopt this python interface
> Patch 3 - 6: emulate uncore/core shared memory region with mmio callbacks and
> add support with this infrastructure.
> 
> I request for comments, suggestions, ideas on getting a scripting interface
> like python added in qemu.
> 
> Balamuruhan S (6):
>   utils/python_api: add scripting interface for Qemu with python lib
>   hw/ppc/pnv_xscom: extend xscom to use python interface
>   hw/ppc/pnv_homer: add homer/occ common area emulation for PowerNV
>   hw/ppc/pnv: initialize and realize homer/occ common area
>   hw/ppc/pnv_xscom: retrieve homer/occ base address from PBA BARs
>   hw/ppc/pnv_homer: add python interface support for homer/occ common
>     area
> 
>  configure                   |  10 +++
>  hw/ppc/Makefile.objs        |   2 +-
>  hw/ppc/pnv.c                |  49 ++++++++++-
>  hw/ppc/pnv_homer.c          | 205 
> ++++++++++++++++++++++++++++++++++++++++++++
>  hw/ppc/pnv_xscom.c          |  59 +++++++++++--
>  include/hw/ppc/pnv.h        |  15 ++++
>  include/hw/ppc/pnv_homer.h  |  41 +++++++++
>  include/sysemu/python_api.h |  30 +++++++
>  include/sysemu/sysemu.h     |   8 ++
>  qemu-options.hx             |  14 +++
>  util/Makefile.objs          |   1 +
>  util/python_api.c           | 100 +++++++++++++++++++++
>  vl.c                        |  66 ++++++++++++++
>  13 files changed, 588 insertions(+), 12 deletions(-)
>  create mode 100644 hw/ppc/pnv_homer.c
>  create mode 100644 include/hw/ppc/pnv_homer.h
>  create mode 100644 include/sysemu/python_api.h
>  create mode 100644 util/python_api.c
>