[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Qemu-devel] [PATCH] posix-aio-compat: fix latency issues
From: |
Frediano Ziglio |
Subject: |
Re: [Qemu-devel] [PATCH] posix-aio-compat: fix latency issues |
Date: |
Mon, 8 Aug 2011 15:21:13 +0200 |
2011/8/8 Avi Kivity <address@hidden>:
> On 08/08/2011 03:49 PM, Frediano Ziglio wrote:
>>
>> 2011/8/8 Avi Kivity<address@hidden>:
>> > In certain circumstances, posix-aio-compat can incur a lot of latency:
>> > - threads are created by vcpu threads, so if vcpu affinity is set,
>> > aio threads inherit vcpu affinity. This can cause many aio threads
>> > to compete for one cpu.
>> > - we can create up to max_threads (64) aio threads in one go; since a
>> > pthread_create can take around 30μs, we have up to 2ms of cpu time
>> > under a global lock.
>> >
>> > Fix by:
>> > - moving thread creation to the main thread, so we inherit the main
>> > thread's affinity instead of the vcpu thread's affinity.
>> > - if a thread is currently being created, and we need to create yet
>> > another thread, let thread being born create the new thread,
>> > reducing
>> > the amount of time we spend under the main thread.
>> > - drop the local lock while creating a thread (we may still hold the
>> > global mutex, though)
>> >
>> > Note this doesn't eliminate latency completely; scheduler artifacts or
>> > lack of host cpu resources can still cause it. We may want
>> > pre-allocated
>> > threads when this cannot be tolerated.
>> >
>> > Thanks to Uli Obergfell of Red Hat for his excellent analysis and
>> > suggestions.
>> >
>> > Signed-off-by: Avi Kivity<address@hidden>
>>
>> Why not calling pthread_attr_setaffinity_np (where available) before
>> thread creation or shed_setaffinity at thread start instead of telling
>> another thread to create a thread for us just to get affinity cleared?
>>
>
> The entire qemu process may be affined to a subset of the host cpus; we
> don't want to break that.
>
> For example:
>
> taskset 0xf0 qemu ....
> (qemu) info cpus
> <pin individual vcpu threads to host cpus>
>
>
Just call sched_getaffinity at program start, save to a global
variable and then set this affinity for io threads.
I didn't use affinity that much but from manual it seems that if you
own process you can set affinity as you like.
IMHO this patch introduce a delay in io thread creation due to posting
thread creation to another thread just to set different affinity.
Frediano