[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Qemu-devel] [PATCH V13 0/5] support MAP_SYNC for memory-backend-fil
From: |
Michael S. Tsirkin |
Subject: |
Re: [Qemu-devel] [PATCH V13 0/5] support MAP_SYNC for memory-backend-file |
Date: |
Thu, 7 Feb 2019 22:01:40 -0500 |
On Fri, Feb 08, 2019 at 06:10:23PM +0800, Zhang, Yi wrote:
> Linux 4.15 introduces a new mmap flag MAP_SYNC, which can be used to
> guarantee the write persistence to mmap'ed files supporting DAX (e.g.,
> files on ext4/xfs file system mounted with '-o dax').
>
> A description of MAP_SYNC and MAP_SHARED_VALIDATE can be found at
> https://patchwork.kernel.org/patch/10028151/
>
> In order to make sure that the file metadata is in sync after a fault
> while we are writing a shared DAX supporting backend files, this
> patch-set enables QEMU to use MAP_SYNC flag for memory-backend-dax-file.
>
> As the DAX vs DMA truncated issue was solved, we refined the code and
> send out this feature for the v5 version.
>
> We will pass MAP_SYNC to mmap(2); if MAP_SYNC is supported and
> 'share=on' & 'pmem=on'.
> Or QEMU will not pass this flag to mmap(2)
Overall this looks reasonable to me.
Besides the documentation nit:
Reviewed-by: Michael S. Tsirkin <address@hidden>
> Test with below cases:
> 1. pmem=on is set, shared=on is set, MAP_SYNC supported:
> a: backend is a dax supporting file.
> 1) start VM1 with options:
> -object
> memory-backend-file,id=nv_be4,share,mem-path=${DAX_FILE_1},size=${DAX_FILE_SIZE_1},align=128M,pmem=on,share=on
> -device nvdimm,id=nv4,memdev=nv_be4,label-size=2M.
>
> 2) start VM2 with options:
> -object
> memory-backend-file,id=nv_be4,share,mem-path=${DAX_FILE_2,size=${DAX_FILE_SIZE_2},align=128M,pmem=on,share=on
> -device nvdimm,id=nv4,memdev=nv_be4,label-size=2M.
>
> 3) live migrate from VM1 to VM2.
>
> 4) Suddly let Host crash or power failure.
>
> 5) check DAX_FILE_1 and DAX_FILE_2, no corrupt.
>
> b: backend is a regular file.
> 1) start with options
> -object
> memory-backend-file,id=nv_be4,share,mem-path=${REG_FILE},size=${REG_FILE_SIZE},align=128M,pmem=on,share=on
> -device nvdimm,id=nv4,memdev=nv_be4,label-size=2M.
>
> will warning "failed to validate with mapping flags: Operation not
> supported"
> FILE_1 and FILE_2 random corrupt.
>
> 2. Other cases:
> FILE_1 and FILE_2 random corrupt.
>
> Changes in V13:
> * 4/5 Micheal: move the inlcude to mmap_alloc.c.
> * 4/5 Micheal: refine the warning message.
> * 5/5 Micheal: refine the Documentations.
>
> Changes in V12:
> * 2/5: Micheal: Update update-linux-headers.sh
> * 3/5: Micheal: Use script update add linux/mman.h
> * 4/5: Pankaj,Micheal: 1) fallback to mmap without
> MAP_SYNC & MAP_SHARED_VALIDATE if sync not supported or failed
> 2) Replace the include with 3/5 added linux/mman.h
> * 5/5: Micheal: Refine the Documentations.
>
> Changes in V11:
> * 1/3: Micheal: Change to just add a bool is_pmem in qemu_ram_mmap.
> * 2/3: Micheal: Fix the compatibility for old kernel.
> * 2/3&3/3: Micheal&Eduardo :Update the behavior below:
> Waning at no-dax and continue without MAP_SYNC.
> Test if fails again for compatibility, then remove the MAP_VALIDATE and
> silently proceed.
>
> Changes in V10:
> * 4/4: refine the document.
> * 3/4: Reviewed-by: Stefano Garzarella <address@hidden>
> * 2/4: refine the commit message, Added MAP_SHARED_VALIDATE.
> * 2/4: Fix the wrong include header
>
> Changes in V9:
> * 1/6: Reviewed-by: Eduardo Habkost <address@hidden>
> * 2/6: New Added: Micheal: use sparse feature define RAM_FLAG.
> since I don't have much knowledge about the sparse feature, @Micheal Could
> you
> add some documentation/commit message on this patch? Thank you very much.
> * 3/6: from 2/5: Eduardo: updated the commit message.
> * 4/6: from 3/5: Micheal: don't ignore MAP_SYNC failures silently.
> * 5/6: from 4/5: Eduardo: updated the commit message.
> * 6/6: from 5/5: Micheal: Drop the sync option, document the MAP_SYNC.
>
> Changes in v8:
> * Micheal: 3/5, remove the duplicated define in the os_dep.h
> * Micheal: 2/5, make type define safety.
> * Micheal: 2/5, fixed the incorrect define MAP_SHARE on qemu_anon_ram_alloc.
> * 4/6 removed, we remove the on/off/auto define of sync, as by now,
> MAP_SYNC only worked with pmem=on.
> * @Micheal, I still reuse the RAM_SYNC flag, it is much straightforward to
> parse
> all the flags in one parameter.
>
> Changes in v7:
> * Micheal: [3,4,6]/6 limited the "sync" flag only on a nvdimm
> backend.(pmem=on)
>
> Changes in v6:
> * Pankaj: 3/7 are squashed with 2/7
> * Pankaj: 7/7 update comments to "consistent filesystem metadata".
> * Pankaj, Igor: 1/7 Added Reviewed-by in patch-1/7
> * Stefan, 4/7 move the include header from "/linux/mman.h" to "osdep.h"
> * Stefan, 5/7 Add missing "munmap"
> * Stefan, 2/7 refine the shared/flag.
>
> Changes in v5:
> * Add patch 1 to fix a memory leak issue.
> * Refine the patch 4-6
> * Remove the patch 3 as we already change the parameter from "shared" to
> "flags"
>
> Changes in v4:
> * Add patch 1-3 to switch some functions to a single 'flags'
> parameters. (Michael S. Tsirkin)
> * v3 patch 1-3 become v4 patch 4-6.
> * Patch 4: move definitions of MAP_SYNC and MAP_SHARED_VALIDATE to a
> new header file under include/standard-headers/linux/. (Michael S. Tsirkin)
> * Patch 6: refine the description of the 'sync' option. (Michael S. Tsirkin)
>
> Changes in v3:
> * Patch 1: add MAP_SHARED_VALIDATE in both sync=on and sync=auto
> cases, and add back the retry mechanism. MAP_SYNC will be ignored
> by Linux kernel 4.15 if MAP_SHARED_VALIDATE is missed.
> * Patch 1: define MAP_SYNC and MAP_SHARED_VALIDATE as 0 on non-Linux
> platforms in order to make qemu_ram_mmap() compile on those platforms.
> * Patch 2&3: include more information in error messages of
> memory-backend in hope to help user to identify the error.
> (Dr. David Alan Gilbert)
> * Patch 3: fix typo in the commit message. (Dr. David Alan Gilbert)
>
> Changes in v2:
> * Add 'sync' option to control the use of MAP_SYNC. (Eduardo Habkost)
> * Remove the unnecessary set of MAP_SHARED_VALIDATE in some cases and
> the retry mechanism in qemu_ram_mmap(). (Michael S. Tsirkin)
> * Move OS dependent definitions of MAP_SYNC and MAP_SHARED_VALIDATE
> to osdep.h. (Michael S. Tsirkin)
>
> Zhang Yi (5):
> util/mmap-alloc: Add a 'is_pmem' parameter to qemu_ram_mmap
> scripts/update-linux-headers: add linux/mman.h
> linux-headers: add linux/mman.h.
> util/mmap-alloc: support MAP_SYNC in qemu_ram_mmap()
> docs: Added MAP_SYNC documentation
>
> Zhang Yi (5):
> util/mmap-alloc: Add a 'is_pmem' parameter to qemu_ram_mmap
> scripts/update-linux-headers: add linux/mman.h
> linux-headers: add linux/mman.h.
> util/mmap-alloc: support MAP_SYNC in qemu_ram_mmap()
> docs: Added MAP_SYNC documentation
>
> docs/nvdimm.txt | 25 ++++++-
> exec.c | 2 +-
> include/qemu/mmap-alloc.h | 21 +++++-
> linux-headers/asm-arm/mman.h | 4 ++
> linux-headers/asm-arm64/mman.h | 1 +
> linux-headers/asm-generic/hugetlb_encode.h | 36 ++++++++++
> linux-headers/asm-generic/mman-common.h | 77 ++++++++++++++++++++
> linux-headers/asm-generic/mman.h | 24 +++++++
> linux-headers/asm-mips/mman.h | 108
> +++++++++++++++++++++++++++++
> linux-headers/asm-powerpc/mman.h | 39 +++++++++++
> linux-headers/asm-s390/mman.h | 1 +
> linux-headers/asm-x86/mman.h | 31 +++++++++
> linux-headers/linux/mman.h | 38 ++++++++++
> qemu-options.hx | 5 ++
> scripts/update-linux-headers.sh | 6 +-
> util/mmap-alloc.c | 51 +++++++++++++-
> util/oslib-posix.c | 2 +-
> 17 files changed, 460 insertions(+), 11 deletions(-)
> create mode 100644 linux-headers/asm-arm/mman.h
> create mode 100644 linux-headers/asm-arm64/mman.h
> create mode 100644 linux-headers/asm-generic/hugetlb_encode.h
> create mode 100644 linux-headers/asm-generic/mman-common.h
> create mode 100644 linux-headers/asm-generic/mman.h
> create mode 100644 linux-headers/asm-mips/mman.h
> create mode 100644 linux-headers/asm-powerpc/mman.h
> create mode 100644 linux-headers/asm-s390/mman.h
> create mode 100644 linux-headers/asm-x86/mman.h
> create mode 100644 linux-headers/linux/mman.h
>
> --
> 2.7.4
- [Qemu-devel] [PATCH V13 0/5] support MAP_SYNC for memory-backend-file, Zhang, Yi, 2019/02/07
- [Qemu-devel] [PATCH V13 1/5] util/mmap-alloc: Add a 'is_pmem' parameter to qemu_ram_mmap, Zhang, Yi, 2019/02/07
- [Qemu-devel] [PATCH V13 2/5] scripts/update-linux-headers: add linux/mman.h, Zhang, Yi, 2019/02/07
- [Qemu-devel] [PATCH V13 3/5] linux-headers: add linux/mman.h., Zhang, Yi, 2019/02/07
- [Qemu-devel] [PATCH V13 4/5] util/mmap-alloc: support MAP_SYNC in qemu_ram_mmap(), Zhang, Yi, 2019/02/07
- [Qemu-devel] [PATCH V13 5/5] docs: Added MAP_SYNC documentation, Zhang, Yi, 2019/02/07
- Re: [Qemu-devel] [PATCH V13 0/5] support MAP_SYNC for memory-backend-file,
Michael S. Tsirkin <=
- Re: [Qemu-devel] [PATCH V13 0/5] support MAP_SYNC for memory-backend-file, Pankaj Gupta, 2019/02/07