Re: [Gluster-devel] Unfair scheduling in unify/AFR

[Anand Avati]

>
> I'm using io-threads on client side. At this time, only one client
> accesses a storage brick at a time (on a given server), so I thought
> io-threads won't help there. But on client side, waiting for a read for
> one thread shouldn't block the whole client (because there can be more
> threads), so I loaded io-threads on the client.


The client does not block for any operation and is completely asynchronous.
The synchronous parts in glusterfs are network writes in tcp (we are working
on a non-blocking socket model design) and posix (linux-aio translator is on
the way). So io-threads would help for reads on the server side alone since
tcp writes of read requests is not really heavy.


> When you copy, are the source and destination files both on the glusterfs
> > mount point?
>
> No, since we are testing pure read performance, and only GlusterFS
> performance. So I copy sparse files over GlsuterFS into /dev/null with
> `dd bs=1M`.
>
> Currently, a single and only thread has a read performance about 540-580
> MB/s. What I would like to see is two threads reading two files from two
> servers, with a performance of at least 540 MB/s *each*.


Are these two dd's from the same machine? If so I suspect that fuse is
becoming a bottleneck. What is your interconnect? 10Gig/E ? You might be
interested in the new booster module which is on the way which will short
cut the IO path directly to the server from the application's address space
(via LD_PRELOAD). You might also want to try the next few TLA commits which
are on the way, which will help parallel file reads going via fuse.


thanks,
avati



-- 
It always takes longer than you expect, even when you take into account
Hofstadter's Law.

-- Hofstadter's Law