[RFC/PATCH v0 12/12] gunyah: Documentation

[Srivatsa Vaddagiri]

Add gunyah.rst that provide some informaiton on how to build and test
'gunyah' accelerator with open-source Gunyah hypervisor.

Signed-off-by: Srivatsa Vaddagiri <quic_svaddagi@quicinc.com>
---
 MAINTAINERS                |   1 +
 docs/system/arm/gunyah.rst | 214 +++++++++++++++++++++++++++++++++++++
 2 files changed, 215 insertions(+)
 create mode 100644 docs/system/arm/gunyah.rst

diff --git a/MAINTAINERS b/MAINTAINERS
index 5a51633d11..f2933a3683 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -518,6 +518,7 @@ F: target/arm/gunyah.c
 F: include/sysemu/gunyah.h
 F: target/arm/arm_gicv3_gunyah.c
 F: include/sysemu/gunyah_int.h
+F: docs/system/arm/gunyah.rst
 
 WHPX CPUs
 M: Sunil Muthuswamy <sunilmut@microsoft.com>
diff --git a/docs/system/arm/gunyah.rst b/docs/system/arm/gunyah.rst
new file mode 100644
index 0000000000..4b3a792310
--- /dev/null
+++ b/docs/system/arm/gunyah.rst
@@ -0,0 +1,214 @@
+'gunyah' accelerator (``gunyah``)
+=================================
+
+Gunyah is a high performance, scalable and flexible hypervisor built for
+demanding battery powered, real-time, safety and security use cases.
+
+The Gunyah Hypervisor open source project provides a reference Type-1 
hypervisor
+configuration suitable for general purpose hosting of multiple trusted and
+dependent VMs. Further information on open-source version of Gunyah Hypervisor
+can be obtained from:
+
+https://github.com/quic/gunyah-hypervisor
+
+To get started with open-source version of Gunyah Hypervisor, refer to the
+instructions available at:
+
+https://github.com/quic/gunyah-support-scripts
+
+Build and testing
+-----------------
+
+Patch and build Gunyah hypervisor
+`````````````````````````````````
+
+Refer to the information present in **Quick Start** guide to get sources for
+Gunyah open-source hypervisor:
+
+https://github.com/quic/gunyah-support-scripts/blob/develop/quickstart.md
+
+Resource Manager VM (RM) is a privileged VM that acts as an extension of Gunyah
+hypervisor. It assists the hypervisor in various tasks related to creation and
+management of VMs. More information on RM is provided at:
+
+https://github.com/quic/gunyah-resource-manager
+
+Instructions provided in **Quick Start** guide will include cloning the sources
+for RM (under 'resource-manager' directory) and hypervisor (under 'hyp'
+directory). These directories will be found inside the docker image created as
+part of steps provided under **Building the Docker Image**.
+
+Before building hypervisor, as described under **Building a Gunyah Hypervisor
+image** section of **Quick Start** guide, apply below changes to hypervisor and
+RM on which 'gunyah' accelerator currently depends. These changes are being
+discussed with maintainers and if accepted this document will be modified
+appropriately.
+
+RM patch (in 'resource-manager' directory):
+
+.. code-block:: bash
+
+       diff --git a/src/vm_creation/vm_creation.c 
b/src/vm_creation/vm_creation.c
+       index df8edfb..b73b37e 100644
+       --- a/src/vm_creation/vm_creation.c
+       +++ b/src/vm_creation/vm_creation.c
+       @@ -510,7 +510,10 @@ process_dtb(vm_t *vm)
+               // Estimate a final dtb size after applying the overlay.
+               size_t original_dtb_size =
+                       util_balign_up(fdt_totalsize(temp_addr), 
sizeof(uint32_t));
+       -       size_t final_dtb_size = original_dtb_size + dtbo_ret.size;
+       +       size_t final_dtb_size = util_balign_up(original_dtb_size + 
dtbo_ret.size, 8);
+
+
+Hypervisor patch (in 'hyp' directory):
+
+.. code-block:: bash
+
+       diff --git a/config/platform/qemu.conf b/config/platform/qemu.conf
+       index bc612f2..9a292a4 100644
+       --- a/config/platform/qemu.conf
+       +++ b/config/platform/qemu.conf
+       @@ -35,7 +35,7 @@ configs HLOS_RAM_FS_BASE=0x40800000
+        configs PLATFORM_HEAP_PRIVATE_SIZE=0x200000
+        configs PLATFORM_RW_DATA_SIZE=0x200000
+        configs PLATFORM_ROOTVM_LMA_BASE=0x80480000U
+       -configs PLATFORM_ROOTVM_LMA_SIZE=0xa0000U
+       +configs PLATFORM_ROOTVM_LMA_SIZE=0x100000U
+        configs PLATFORM_PHYS_ADDRESS_BITS=36
+        configs PLATFORM_VM_ADDRESS_SPACE_BITS=36
+        configs PLATFORM_PGTABLE_4K_GRANULE=1
+
+Build Gunyah hypervisor (including RM) as per the instructions provided in
+**Quick Start** guide, viz:
+
+.. code-block:: bash
+
+        cd ~/share
+        build-gunyah.sh qemu
+
+
+Patch and build Linux kernel
+````````````````````````````
+
+**Building the Docker Image** step of **Quick Start** guide would have cloned
+the sources for Linux kernel (under '~/linux/src/linux' directory in docker
+image) and also applied below patches.
+
+https://lore.kernel.org/lkml/20230613172054.3959700-1-quic_eberman@quicinc.com/
+
+Above patches provide a kernel driver that interacts with Gunyah hypervisor and
+assists a VMM (like Qemu) in creation and management of a VM. Apply below patch
+to Linux kernel, as the 'gunyah' accelerator in Qemu depends on it.
+
+.. code-block:: bash
+
+       diff --git a/drivers/virt/gunyah/gunyah_ioeventfd.c 
b/drivers/virt/gunyah/gunyah_ioeventfd.c
+       index 5b1b9fd9a..7bfb67800 100644
+       --- a/drivers/virt/gunyah/gunyah_ioeventfd.c
+       +++ b/drivers/virt/gunyah/gunyah_ioeventfd.c
+       @@ -88,6 +88,7 @@ static long gh_ioeventfd_bind(struct 
gh_vm_function_instance *f)
+               }
+               iofd->io_handler.addr = args->addr;
+               iofd->io_handler.ops = &io_ops;
+       +       iofd->io_handler.fd = args->fd;
+
+               ret = gh_vm_add_io_handler(f->ghvm, &iofd->io_handler);
+               if (ret)
+       @@ -120,7 +121,7 @@ static bool gh_ioevent_compare(const struct 
gh_vm_function_instance *f,
+               if (sizeof(*other) != size)
+                       return false;
+
+       -       return instance->addr == other->addr;
+       +       return instance->addr == other->addr && instance->fd == 
other->fd;
+        }
+
+        DECLARE_GH_VM_FUNCTION_INIT(ioeventfd, GH_FN_IOEVENTFD, 3,
+       diff --git a/drivers/virt/gunyah/vm_mgr.c b/drivers/virt/gunyah/vm_mgr.c
+       index afea49238..c4604d4e9 100644
+       --- a/drivers/virt/gunyah/vm_mgr.c
+       +++ b/drivers/virt/gunyah/vm_mgr.c
+       @@ -291,6 +291,10 @@ static int _gh_vm_io_handler_compare(const struct 
rb_node *node, const struct rb
+                       return -1;
+               if (n->len > p->len)
+                       return 1;
+       +       if (n->fd > p->fd)
+       +               return 1;
+       +       if (n->fd < p->fd)
+       +               return -1;
+               /* one of the io handlers doesn't have datamatch and the other 
does.
+                * For purposes of comparison, that makes them identical since 
the
+                * one that doesn't have datamatch will cover the same handler 
that
+       diff --git a/include/linux/gunyah_vm_mgr.h 
b/include/linux/gunyah_vm_mgr.h
+       index 527e94624..b2e95d02c 100644
+       --- a/include/linux/gunyah_vm_mgr.h
+       +++ b/include/linux/gunyah_vm_mgr.h
+       @@ -136,6 +136,7 @@ struct gh_vm_io_handler {
+               bool datamatch;
+               u8 len;
+               u64 data;
+       +       int fd;
+               struct gh_vm_io_handler_ops *ops;
+        };
+
+Run 'build-linux.sh' inside the docker container to build Linux kernel, which
+provides the image for host kernel.
+
+.. code-block:: bash
+
+        $ cd ~/
+        $ build-linux.sh
+
+
+This will result in Linux kernel binary (Image) and an initrd image 
(initrd.img)
+being created and copied to '~/tools/qemu/imgs' directory. Those images can be
+used for both host kernel as well as for a VM.
+
+
+Configure and build Qemu
+````````````````````````
+
+Apply the proposed patches for 'gunyah' accelerator support in Qemu and build
+it.
+
+.. code-block:: bash
+
+        $ ./configure --target-list=aarch64-softmmu --enable-debug 
--enable-gunyah --static
+        $ make -j4
+
+Incorporate Qemu image in root disk
+```````````````````````````````````
+
+Copy over the resulting Qemu static binary ('qemu-system-aarch64') into the
+docker image. In addition, copy 'efi-virtio.rom' and 'en-us' (from Qemu
+repository) into the docker image.
+
+Follow the instructions under **SVM booting Linux** to prepare a virtual disk
+that can host the required images. Since Qemu is going to be used in place of
+CrosVM, ignore the CrosVM related steps, and instead copy these files to
+'~/share/docker-share/rootfs/usr/gunyah' folder before running
+'~/utils/bldextfs.sh' script to build root image.
+
+qemu-system-aarch64, efi-virtio.rom, en-us, Image, initrd.img
+
+The root disk created ('rootfs-extfs-disk.img') will have above files under
+/usr/gunyah folder.
+
+Running VM
+``````````
+
+.. code-block:: bash
+
+        $ cd /usr/gunyah
+        $ ./qemu-system-aarch64 -cpu cortex-a57 -nographic -m 256M --accel 
gunyah \
+        -machine virt,highmem=off -initrd ./initrd.img -append \
+        "rw root=/dev/ram rdinit=/sbin/init earlyprintk=serial panic=0"
+        -kernel ./Image
+
+Limitations
+-----------
+
+Below features are not yet supported.
+
+* SMP (PSCI support needs to be added)
+* virtio-pci (support for which in Qemu seems to rely heavily on KVM, which
+  needs to be made multi-hypervisor friendly).
-- 
2.25.1