Re: [Qemu-block] RFC: Status of the intermediate block streaming work

[Stefan Hajnoczi]

On Thu, Dec 10, 2015 at 06:07:54PM +0100, Alberto Garcia wrote:
> Hi all,
> 
> I have been retaking my work on the intermediate block streaming
> patches. Here's a summary of this feature and the current problems:
> 
> The goal is to be able to perform a 'block-stream' operation on a node
> different from the active layer:
> 
> [A] <- [B] <- [C] <- [D] <- [E]
> 
> In this case, we would be able to copy the data from [C] into [D],
> which would result in the following graph:
> 
> [A] <- [B] <- [D] <- [E]
> 
> The block job would be created in [D], and all nodes that are not
> involved in the operation would be free for other block jobs. That
> would allow us to run several block-stream commands in parallel, and
> in the example above we would be able to copy [A] into [B] and [C]
> into [D] in parallel, which would result in this chain:
> 
> [B] <- [D] <- [E]
> 
> The code has been ready for more than half a year, but there are
> currently two problems that prevent it from working:
> 
>   1) The destination images need to be reopened in read-write
>      mode. This has the side effect of reopening its whole backing
>      chain in read-only mode. If there's a block job writing in one of
>      those images it will fail.
> 
>      In the example above, if we stream [A] into [B] and then try to
>      stream [C] into [D], then [B] will switch to read-only mode and
>      the first operation will fail.
> 
>      The problem is described here:
> 
>      https://lists.gnu.org/archive/html/qemu-block/2015-10/msg01042.html
> 
>      As I understand it, Kevin's work on bdrv_reopen() will fix that.

I think you are asking Kevin about the status of this work.

Kevin?

>   2) When a block-stream operation calls bdrv_reopen(), it drains all
>      images. This can trigger the completion of a different
>      block-stream operation, that will remove the intermediate images
>      from the chain. When the original bdrv_reopen() continues, it
>      still contains pointers to the images that have just been
>      removed, crashing QEMU.
> 
>      The problem is described here:
> 
>      https://lists.gnu.org/archive/html/qemu-block/2015-06/msg00701.html
> 
>      I'm still not sure about how to deal with this. I've been taking
>      a look at the bdrv_drained_begin/end() API, but as I understand
>      it it prevents requests from a different AioContext.  Since all
>      BDS in the same chain share the same context it does not really
>      help here.

You are right, bdrv_drained_begin/end() does not solve this problem.

bdrv_reopen_multiple() claims to be atomic but can call aio_poll().

I think blockjobs need the ability to quiesce or attach/detach from the
AioContext (like BlockDriverStates).  That way reopen can be made truly
atomic.

The solution isn't clear in my mind.  Adding a quiesce operation to all
blockjobs is probably not that hard to do.  I wonder whether that is a
nice long-term solution though...

> Since the ability to run two jobs in parallel is of secondary
> importance to me, I'm considering to disallow it by locking the whole
> chain until these two issues have been solved, but any suggestions are
> more than welcome.
> 
> Thanks,
> 
> Berto
>

signature.asc
Description: PGP signature