Re: [PATCH for-5.1] file-posix: Mitigate file fragmentation with extent size hints

[Eric Blake]

On 7/7/20 9:23 AM, Kevin Wolf wrote:
> Espeically when O_DIRECT is used with image files so that the page cache

Especially

> indirection can't cause a merge of allocating requests, the file will
> fragment on the file system layer, with a potentially very small
> fragment size (this depends on the requests the guest sent).
>
> On Linux, fragmentation can be reduced by setting an extent size hint
> when creating the file (at least on XFS, it can't be set any more after
> the first extent has been allocated), basically giving raw files a
> "cluster size" for allocation.
>
> This adds an create option to set the extent size hint, and changes the
> default from not setting a hint to setting it to 1 MB. The main reason
> why qcow2 defaults to smaller cluster sizes is that COW becomes more
> expensive, which is not an issue with raw files, so we can choose a
> larger file. The tradeoff here is only potentially wasted disk space.
>
> For qcow2 (or other image formats) over file-posix, the advantage should
> even be greater because they grow sequentially without leaving holes, so
> there won't be wasted space. Setting even larger extent size hints for
> such images may make sense. This can be done with the new option, but
> let's keep the default conservative for now.
>
> The effect is very visible with a test that intentionally creates a
> badly fragmented file with qemu-img bench (the time difference while
> creating the file is already remarkable) and then looks at the number of
> extents and the take a simple "qemu-img map" takes.

Cool!

> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> ---
>   qapi/block-core.json      | 11 ++++++----
>   include/block/block_int.h |  1 +
>   block/file-posix.c        | 42 +++++++++++++++++++++++++++++++++++++++
>   3 files changed, 50 insertions(+), 4 deletions(-)

> +#ifdef FS_IOC_FSSETXATTR
> +    /*
> +     * Try to set the extent size hint. Failure is not fatal, and a warning is
> +     * only printed if the option was explicitly specified.
> +     */
> +    {
> +        struct fsxattr attr;
> +        result = ioctl(fd, FS_IOC_FSGETXATTR, &attr);
> +        if (result == 0) {
> +            attr.fsx_xflags |= FS_XFLAG_EXTSIZE;
> +            attr.fsx_extsize = file_opts->extent_size_hint;
> +            result = ioctl(fd, FS_IOC_FSSETXATTR, &attr);
> +        }
> +        if (result < 0 && file_opts->has_extent_size_hint) {
> +            warn_report("Failed to set extent size hint: %s",
> +                        strerror(errno));
> +        }
> +    }
> +#endif

That's a neat ioctl to learn.

Reviewed-by: Eric Blake <eblake@redhat.com>

--
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org