[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Qemu-ppc] [PATCH qemu v10 14/14] spapr_pci/spapr_pci_vfio: Support
From: |
David Gibson |
Subject: |
Re: [Qemu-ppc] [PATCH qemu v10 14/14] spapr_pci/spapr_pci_vfio: Support Dynamic DMA Windows (DDW) |
Date: |
Tue, 7 Jul 2015 14:58:07 +1000 |
User-agent: |
Mutt/1.5.23 (2014-03-12) |
On Mon, Jul 06, 2015 at 12:11:10PM +1000, Alexey Kardashevskiy wrote:
> This adds support for Dynamic DMA Windows (DDW) option defined by
> the SPAPR specification which allows to have additional DMA window(s)
>
> This implements DDW for emulated and VFIO devices. As all TCE root regions
> are mapped at 0 and 64bit long (and actual tables are child regions),
> this replaces memory_region_add_subregion() with _overlap() to make
> QEMU memory API happy.
>
> This reserves RTAS token numbers for DDW calls.
>
> This implements helpers to interact with VFIO kernel interface.
>
> This changes the TCE table migration descriptor to support dynamic
> tables as from now on, PHB will create as many stub TCE table objects
> as PHB can possibly support but not all of them might be initialized at
> the time of migration because DDW might or might not be requested by
> the guest.
>
> The "ddw" property is enabled by default on a PHB but for compatibility
> the pseries-2.3 machine and older disable it.
>
> This implements DDW for VFIO. The host kernel support is required.
> This adds a "levels" property to PHB to control the number of levels
> in the actual TCE table allocated by the host kernel, 0 is the default
> value to tell QEMU to calculate the correct value. Current hardware
> supports up to 5 levels.
>
> The existing linux guests try creating one additional huge DMA window
> with 64K or 16MB pages and map the entire guest RAM to. If succeeded,
> the guest switches to dma_direct_ops and never calls TCE hypercalls
> (H_PUT_TCE,...) again. This enables VFIO devices to use the entire RAM
> and not waste time on map/unmap later. This adds a "dma64_win_addr"
> property which is a bus address for the 64bit window and by default
> set to 0x800.0000.0000.0000 as this is what the modern POWER8 hardware
> uses and this allows having emulated and VFIO devices on the same bus.
>
> This adds 4 RTAS handlers:
> * ibm,query-pe-dma-window
> * ibm,create-pe-dma-window
> * ibm,remove-pe-dma-window
> * ibm,reset-pe-dma-window
> These are registered from type_init() callback.
>
> These RTAS handlers are implemented in a separate file to avoid polluting
> spapr_iommu.c with PCI.
> diff --git a/trace-events b/trace-events
> index b300e94..a1234dd 100644
> --- a/trace-events
> +++ b/trace-events
> @@ -1302,6 +1302,8 @@ spapr_pci_lsi_set(const char *busname, int pin,
> uint32_t irq) "%s PIN%d IRQ %u"
> spapr_pci_msi_retry(unsigned config_addr, unsigned req_num, unsigned
> max_irqs) "Guest device at %x asked %u, have only %u"
> spapr_pci_dma_update(uint64_t liobn, long ret) "liobn=%"PRIx64" ret=%ld"
> spapr_pci_dma_realloc_update(uint64_t liobn, long ret) "liobn=%"PRIx64"
> tcet=%ld"
> +spapr_pci_vfio_init_window(int ps, uint64_t ws, uint64_t off)
> "pageshift=0x%x winsize=0x%"PRIx64" offset=0x%"PRIx64
> +spapr_pci_vfio_remove_window(uint64_t off) "offset=%"PRIx64
>
> # hw/pci/pci.c
> pci_update_mappings_del(void *d, uint32_t bus, uint32_t func, uint32_t slot,
> int bar, uint64_t addr, uint64_t size) "d=%p %02x:%02x.%x
> %d,%#"PRIx64"+%#"PRIx64
> @@ -1365,6 +1367,10 @@ spapr_iommu_pci_indirect(uint64_t liobn, uint64_t
> ioba, uint64_t tce, uint64_t i
> spapr_iommu_pci_stuff(uint64_t liobn, uint64_t ioba, uint64_t tce_value,
> uint64_t npages, uint64_t ret) "liobn=%"PRIx64" ioba=0x%"PRIx64"
> tcevalue=0x%"PRIx64" npages=%"PRId64" ret=%"PRId64
> spapr_iommu_xlate(uint64_t liobn, uint64_t ioba, uint64_t tce, unsigned
> perm, unsigned pgsize) "liobn=%"PRIx64" 0x%"PRIx64" -> 0x%"PRIx64" perm=%u
> mask=%x"
> spapr_iommu_alloc_table(uint64_t liobn, void *table, int fd)
> "liobn=%"PRIx64" table=%p fd=%d"
> +spapr_iommu_ddw_query(uint64_t buid, uint32_t cfgaddr, unsigned wa, uint64_t
> win_size, uint32_t pgmask) "buid=%"PRIx64" addr=%"PRIx32", %u windows
> available, max window size=%"PRIx64", mask=%"PRIx32
Turns out the dtrace trace backend barfs on the "long long" here :(
$ ./configure --target-list=ppc64-softmmu --enable-trace-backends=dtrace
[...]
$ make
GEN config-host.h
GEN trace/generated-tracers.h
GEN trace/generated-tracers-dtrace.dtrace
GEN trace/generated-tracers-dtrace.h
Warning: /bin/dtrace:trace/generated-tracers-dtrace.dtrace:2212: syntax error
near:
probe spapr_iommu_ddw_query
Warning: Proceeding as if --no-pyparsing was given.
GEN trace/generated-tcg-tracers.h
GEN trace/generated-helpers-wrappers.h
GEN trace/generated-helpers.h
CC trace/generated-events.o
GEN trace/generated-tracers-dtrace.o
Warning: /bin/dtrace:trace/generated-tracers-dtrace.dtrace:2212: syntax error
near:
probe spapr_iommu_ddw_query
Warning: Proceeding as if --no-pyparsing was given.
--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson
pgpexKf4aX0dH.pgp
Description: PGP signature
- [Qemu-ppc] [PATCH qemu v10 09/14] spapr_vfio_pci: Remove redundant spapr-pci-vfio-host-bridge, (continued)
[Qemu-ppc] [PATCH qemu v10 05/14] spapr_iommu: Introduce "enabled" state for TCE table, Alexey Kardashevskiy, 2015/07/05
Re: [Qemu-ppc] [PATCH qemu v10 00/14] spapr: vfio: Enable Dynamic DMA windows (DDW), David Gibson, 2015/07/06
Re: [Qemu-ppc] [Qemu-devel] [PATCH qemu v10 00/14] spapr: vfio: Enable Dynamic DMA windows (DDW), Thomas Huth, 2015/07/06