[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Qemu-devel] Re: [PATCH][v2] Align file accesses with cache=off (O_D
From: |
Avi Kivity |
Subject: |
Re: [Qemu-devel] Re: [PATCH][v2] Align file accesses with cache=off (O_DIRECT) |
Date: |
Wed, 21 May 2008 17:57:08 +0300 |
User-agent: |
Thunderbird 2.0.0.14 (X11/20080501) |
Anthony Liguori wrote:
Avi Kivity wrote:
Anthony Liguori wrote:
"cached" is not a terribly accurate term. O_DIRECT avoids the host
page cache but it doesn't guarantee that the disk is using
write-through. For that, you need to use hdparm.
O_SYNC basically turns the host page cache into a write-through
cache. In terms of data integrity, the only question that matters
is whether you're misleading the guest into thinking data is on the
disk when it isn't. Both O_DIRECT and O_SYNC accomplish this.
If you just are concerned with data integrity, O_SYNC is probably
better because you get the benefits of host caching. O_DIRECT is
really for circumstances where you know that using the host page
cache is going to reduce performance.
In one specific circumstance O_SYNC has data integrity problems:
shared disks with guests running on different hosts (or even a guest
on one host, sharing a disk with another host). In these cases, two
reads can return different values without an intervening write.
Are you assuming the underlying disk sharing protocol does not keep
the page cache coherent?
I was assuming access to a raw partition. But yes, with a cluster file
system this objection goes away.
At a significant cost, though. You're now running two cache coherency
protocols on top of each other.
In the general case, O_DIRECT gives better performance. It avoids
copying from the host pagecache to guest memory, and if you have
spare memory to benefit from caching, give it to the guest; the
nearer to the data consumer the cache is, the faster it performs.
This assumes the only thing you're running on the machine is VMs. If
you're just running one VM on your desktop, it is probably preferable
to go through the host page cache. In particular, the host page cache
can be automatically aged and adjusted whereas, in general, you cannot
reduce the guests page cache size.
Agreed. For casual uses, O_DIRECT is overkill. It does get rid of the
data copies, though.
In one specific case O_SYNC (or regular read/write cached operation)
is better, rebooting the same guest over and over, as the guest cache
is flushed on reboot. Not a very interesting case, but unfortunately
one that is very visible.
(if you have a backing file for a COW disk, then opening the backing
file without O_DIRECT may be a good idea too, as the file can be
shared among many guests).
FWIW, we really only need to use O_SYNC when the guest has disabled
write-back. I think we should do that unconditionally too as it's an
issue of correctness.
If the guest is used for non critical applications (like testing distro
installers), then it's just a slowdown.
Even if the guest did not disable disk writeback, pagecache and disk
write caches have vastly different characteristics, so I think we should
set O_SYNC there as well.
Here's a summary of the use cases I saw so far:
- casual use, no critical data: write back cache
- backing file shared among many guests: read-only, cached
- desktop system, but don't lose my data: O_SYNC
(significant resources on the host)
- dedicated virtualization engine: O_DIRECT
(most host resources assigned to guests)
--
error compiling committee.c: too many arguments to function
- Re: [Qemu-devel] Re: [PATCH][v2] Align file accesses with cache=off (O_DIRECT), (continued)
- Re: [Qemu-devel] Re: [PATCH][v2] Align file accesses with cache=off (O_DIRECT), Kevin Wolf, 2008/05/21
- Re: [Qemu-devel] Re: [PATCH][v2] Align file accesses with cache=off (O_DIRECT), Paul Brook, 2008/05/20
- Re: [Qemu-devel] Re: [PATCH][v2] Align file accesses with cache=off (O_DIRECT), Anthony Liguori, 2008/05/20
- Re: [Qemu-devel] Re: [PATCH][v2] Align file accesses with cache=off (O_DIRECT), Kevin Wolf, 2008/05/21
- Re: [Qemu-devel] Re: [PATCH][v2] Align file accesses with cache=off (O_DIRECT), Jamie Lokier, 2008/05/21
- Re: [Qemu-devel] Re: [PATCH][v2] Align file accesses with cache=off (O_DIRECT), Avi Kivity, 2008/05/21
- Re: [Qemu-devel] Re: [PATCH][v2] Align file accesses with cache=off (O_DIRECT), Jamie Lokier, 2008/05/21
- Re: [Qemu-devel] Re: [PATCH][v2] Align file accesses with cache=off (O_DIRECT), Anthony Liguori, 2008/05/21
- Re: [Qemu-devel] Re: [PATCH][v2] Align file accesses with cache=off (O_DIRECT), Avi Kivity, 2008/05/21
- Re: [Qemu-devel] Re: [PATCH][v2] Align file accesses with cache=off (O_DIRECT), Anthony Liguori, 2008/05/21
- Re: [Qemu-devel] Re: [PATCH][v2] Align file accesses with cache=off (O_DIRECT),
Avi Kivity <=
- Re: [Qemu-devel] Re: [PATCH][v2] Align file accesses with cache=off (O_DIRECT), Jamie Lokier, 2008/05/21
- Re: [Qemu-devel] Re: [PATCH][v2] Align file accesses with cache=off (O_DIRECT), Anthony Liguori, 2008/05/21
- Re: [Qemu-devel] Re: [PATCH][v2] Align file accesses with cache=off (O_DIRECT), Jamie Lokier, 2008/05/21
- Re: [Qemu-devel] Re: [PATCH][v2] Align file accesses with cache=off (O_DIRECT), Avi Kivity, 2008/05/21
- Re: [Qemu-devel] Re: [PATCH][v2] Align file accesses with cache=off (O_DIRECT), Andrea Arcangeli, 2008/05/21
- Re: [Qemu-devel] Re: [PATCH][v2] Align file accesses with cache=off (O_DIRECT), Avi Kivity, 2008/05/21
- Re: [Qemu-devel] Re: [PATCH][v2] Align file accesses with cache=off (O_DIRECT), Andrea Arcangeli, 2008/05/21
- Re: [Qemu-devel] Re: [PATCH][v2] Align file accesses with cache=off (O_DIRECT), Anthony Liguori, 2008/05/21
- Re: [Qemu-devel] Re: [PATCH][v2] Align file accesses with cache=off (O_DIRECT), Andrea Arcangeli, 2008/05/21
- Re: [Qemu-devel] Re: [PATCH][v2] Align file accesses with cache=off (O_DIRECT), Anthony Liguori, 2008/05/21