Re: [PATCH 00/18] block: Allow exporting BDSs via FUSE

[Max Reitz]

On 20.12.19 11:08, Stefan Hajnoczi wrote:
> On Thu, Dec 19, 2019 at 03:38:00PM +0100, Max Reitz wrote:
>> Preamble: This series is based on a combination of my (current) block
>> branch and “iotests: Minor fixes”.  I’ve pushed it here:
>>
>>   https://git.xanclic.moe/XanClic/qemu fuse-exports-v1
>>
>> (The base is on fuse-exports-v1-base.)
>>
>>
>> Hi,
>>
>> Ever since I found out that you can mount FUSE filesystems on regular
>> files (not just directories), I had the idea of adding FUSE block
>> exports to qemu where you can export block nodes as raw images.  The
>> best thing is that you’d be able to mount an image on itself, so
>> whatever format it may be in, qemu lets it appear as a raw image (and
>> you can then use regular tools like dd on it).
>>
>> I started with some concept of a qemu-blkfuse daemon (similar to
>> qemu-nbd), but never sent patches, for two reasons: (1) Performance was
>> not good, (2) it didn’t seem right, for some reason.
>>
>> Now Kevin is proposing a storage daemon for multiple export types like
>> NBD, and he also mentioned FUSE (while knowing of my previous attempts).
>> Now it does seem right to add FUSE exports to qemu, but only in the form
>> of some module with a proper QAPI/QMP binding.
>>
>> Performance is still quite bad, but who cares.  We can always improve
>> it, if the need arises.
>>
>>
>> This series does the following:
>>
>> First, add the FUSE export module (block/fuse.c) that implements the
>> basic file access functions.  (Note that you need libfuse 3.8.0 or later
>> for SEEK_HOLE/SEEK_DATA.)
>>
>> Second, it allows using FUSE exports as a protocol in the iotests and
>> makes many iotests work with it.  (The file node is exported by a
>> background qemu instance to $SOCK_DIR.)
>> Note that I only ran raw and qcow2 on it; I’m sure other formats
>> currently have some failing tests.
>>
>> This gives us a lot of coverage for, well, not free (it does take ten
>> patches), but for cheap; but there are still some more specialized
>> things we want to test, so third and last, this series adds an iotest
>> dedicated to FUSE exports.
>>
>>
>> Final rather important notice: I didn’t really run the iotests with this
>> yet.  I wanted to, but they appear rather broken on current master,
>> actually.  I’m not yet sure whether that’s because something in my setup
>> broke in the last two weeks, or because there’s quite something broken
>> in master (it does look like there are a couple things broken in master
>> currently).
>>
>>
>> Max Reitz (18):
>>   configure: Detect libfuse
>>   fuse: Allow exporting BDSs via FUSE
>>   fuse: Implement standard FUSE operations
>>   fuse: Add fuse-export-remove
>>   fuse: Allow growable exports
>>   fuse: (Partially) implement fallocate()
>>   fuse: Implement hole detection through lseek
>>   iotests: Do not needlessly filter _make_test_img
>>   iotests: Do not pipe _make_test_img
>>   iotests: Use convert -n in some cases
>>   iotests: Avoid renaming images
>>   iotests: Derive image names from $TEST_IMG
>>   iotests/091: Use _cleanup_qemu instad of "wait"
>>   iotests: Restrict some Python tests to file
>>   iotests: Let _make_test_img guess $TEST_IMG_FILE
>>   iotests: Allow testing FUSE exports
>>   iotests: Enable fuse for many tests
>>   iotests/281: Add test for FUSE exports
>>
>>  block.c                          |   4 +
>>  block/Makefile.objs              |   3 +
>>  block/fuse.c                     | 668 +++++++++++++++++++++++++++++++
>>  configure                        |  68 ++++
>>  include/block/fuse.h             |  24 ++
>>  qapi/block.json                  |  42 ++
>>  tests/qemu-iotests/013           |   9 +-
>>  tests/qemu-iotests/013.out       |   3 +-
>>  tests/qemu-iotests/018           |   5 +-
>>  tests/qemu-iotests/018.out       |   1 +
>>  tests/qemu-iotests/020           |   2 +-
>>  tests/qemu-iotests/025           |   2 +-
>>  tests/qemu-iotests/026           |   2 +-
>>  tests/qemu-iotests/028           |  16 +-
>>  tests/qemu-iotests/028.out       |   3 +
>>  tests/qemu-iotests/031           |   2 +-
>>  tests/qemu-iotests/034           |   2 +-
>>  tests/qemu-iotests/036           |   2 +-
>>  tests/qemu-iotests/037           |   2 +-
>>  tests/qemu-iotests/038           |   2 +-
>>  tests/qemu-iotests/039           |   2 +-
>>  tests/qemu-iotests/046           |   7 +-
>>  tests/qemu-iotests/046.out       |   2 +-
>>  tests/qemu-iotests/050           |   2 +-
>>  tests/qemu-iotests/054           |   2 +-
>>  tests/qemu-iotests/060           |   2 +-
>>  tests/qemu-iotests/071           |  21 +-
>>  tests/qemu-iotests/072           |   5 +-
>>  tests/qemu-iotests/072.out       |   1 +
>>  tests/qemu-iotests/079           |   2 +-
>>  tests/qemu-iotests/080           |   2 +-
>>  tests/qemu-iotests/089           |   5 +-
>>  tests/qemu-iotests/089.out       |   1 +
>>  tests/qemu-iotests/090           |   2 +-
>>  tests/qemu-iotests/091           |   5 +-
>>  tests/qemu-iotests/095           |   2 +-
>>  tests/qemu-iotests/097           |   2 +-
>>  tests/qemu-iotests/098           |   2 +-
>>  tests/qemu-iotests/102           |   2 +-
>>  tests/qemu-iotests/103           |   2 +-
>>  tests/qemu-iotests/106           |   2 +-
>>  tests/qemu-iotests/107           |   2 +-
>>  tests/qemu-iotests/108           |   2 +-
>>  tests/qemu-iotests/111           |   2 +-
>>  tests/qemu-iotests/112           |   2 +-
>>  tests/qemu-iotests/115           |   2 +-
>>  tests/qemu-iotests/117           |   2 +-
>>  tests/qemu-iotests/120           |   2 +-
>>  tests/qemu-iotests/121           |   2 +-
>>  tests/qemu-iotests/127           |   2 +-
>>  tests/qemu-iotests/133           |   2 +-
>>  tests/qemu-iotests/137           |   2 +-
>>  tests/qemu-iotests/138           |   2 +-
>>  tests/qemu-iotests/140           |   2 +-
>>  tests/qemu-iotests/154           |   2 +-
>>  tests/qemu-iotests/161           |  14 +-
>>  tests/qemu-iotests/171           |   2 +-
>>  tests/qemu-iotests/174           |  10 +-
>>  tests/qemu-iotests/175           |   8 +-
>>  tests/qemu-iotests/176           |   2 +-
>>  tests/qemu-iotests/177           |   2 +-
>>  tests/qemu-iotests/179           |   2 +-
>>  tests/qemu-iotests/183           |   2 +-
>>  tests/qemu-iotests/186           |   2 +-
>>  tests/qemu-iotests/187           |   2 +-
>>  tests/qemu-iotests/191           |   2 +-
>>  tests/qemu-iotests/195           |   2 +-
>>  tests/qemu-iotests/200           |   5 +-
>>  tests/qemu-iotests/200.out       |   4 +-
>>  tests/qemu-iotests/204           |   2 +-
>>  tests/qemu-iotests/206           |   1 +
>>  tests/qemu-iotests/214           |   2 +-
>>  tests/qemu-iotests/217           |   2 +-
>>  tests/qemu-iotests/220           |   2 +-
>>  tests/qemu-iotests/221           |   2 +-
>>  tests/qemu-iotests/229           |   5 +-
>>  tests/qemu-iotests/229.out       |   6 +-
>>  tests/qemu-iotests/242           |   1 +
>>  tests/qemu-iotests/247           |   2 +-
>>  tests/qemu-iotests/249           |   8 +-
>>  tests/qemu-iotests/250           |   2 +-
>>  tests/qemu-iotests/252           |   2 +-
>>  tests/qemu-iotests/265           |   2 +-
>>  tests/qemu-iotests/268           |   2 +-
>>  tests/qemu-iotests/272           |   2 +-
>>  tests/qemu-iotests/273           |   2 +-
>>  tests/qemu-iotests/279           |   2 +-
>>  tests/qemu-iotests/281           | 328 +++++++++++++++
>>  tests/qemu-iotests/281.out       |  92 +++++
>>  tests/qemu-iotests/check         |   6 +
>>  tests/qemu-iotests/common.filter |   5 +-
>>  tests/qemu-iotests/common.rc     | 190 ++++++++-
>>  tests/qemu-iotests/group         |   1 +
>>  93 files changed, 1571 insertions(+), 120 deletions(-)
>>  create mode 100644 block/fuse.c
>>  create mode 100644 include/block/fuse.h
>>  create mode 100755 tests/qemu-iotests/281
>>  create mode 100644 tests/qemu-iotests/281.out
> 
> A lot of qemu-iotests changes but the actual FUSE code is small and well
> worth it for the flexibility and convenience that this feature brings.
> Nice!

Good that you like it! :-)

> Please send a follow-up patch that adds a qemu(1) -blockdev
> 'Driver-specific options for "fuse"' documentation section.

What exactly do you mean?  This is not a block driver, so it doesn’t
work as part of -blockdev.  Currently, it can only be used through QMP
(fuse-export-add/fuse-export-remove).

I placed it into block/ because that just seemed like the least bad
place to me (apart from creating a new top-level directory like nbd has)
– and also because we already have quite some few non-driver files in
block/ (io.c, the jobs (where some got drivers only rather recently),
accounting.c, ...).

Max

signature.asc
Description: OpenPGP digital signature