Re: Open qcow2 on multiple hosts simultaneously.

[kvaps]

> QEMU's qcow2 implementation only supports one writer. Live migration is
> a slight exception, it takes care that only one host writes at any given
> time and also drops caches to avoid stale metadata during migration.

Thank you for clarifying this. Is there any command which I can send
via QMP to drop the caches. Or should I just reopen the file?

> QEMU's "raw" block driver has offset/size parameters, so a ReadWriteMany
> LUN can be divided into smaller block devices and each host can run its
> own qcow2 inside. The problem with this approach is fragmentation and
> ultimately there must be a component in the system that is aware of
> where each host has allocated its slice of the LUN.
>
> If you implement something from scratch, then it might be possible to
> take advantage of Kubernetes. For example, a Custom Resource Definition
> that describes extents allocated from a shared LUN. Space allocation
> could be persisted at the k8s cluster level via this CRD so that you
> don't need to reimplement clustering yourself. The CSI plugin constructs
> qemu-storage-daemon command-lines from the extent information. That way,
> each node in the cluster can run qemu-storage-daemon on the
> ReadWriteMany LUN and coordinate which extents are allocated for a given
> slice.

Yeah, I was considering idea of using offsets on pure LUN. Removing
cluster LVM from this scheme could simplify it a lot. But problem is
that resize will not work. Of course you can allocate a new qcow2 in
chain for every resize operation, but in that case user rewrites will
only allocate the new space instead of rewriting already existing
data.

> So there is definitely something missing if you want to coordinate
> between hosts sharing the same ReadWriteMany LUN. I'm inclined to
> investigate existing solutions like pNFS or cLVM instead of reinventing
> this for QEMU, because I suspect it seems simple at first but actually
> involves a lot of work.

Thanks for pointing this out. In previous mail I mentioned that I'm
going to use lvmlockd with sanlock backend to coordinate LVM metadata
changes.

Best Regards,
Andrei Kvapil