Re: [Gluster-devel] Re: AFR load-balancing

[Kevan Benson]

Jerker Nyberg wrote:
> > > Writes are slow, (since a client need to write to all servers, but 
> > > perhaps is it possible to stack the afrs on the server side and let 
> > > the servers do the replicating when writing... Hmmm...)
> >
> > Are you using write-behind?
>
> Yes. But it was something else I had in mind. I'll try to explain.
>
> When reading from many nodes an afr-translator on the client with say 4 
> subvolumes are fine, since the data may be read directly from a single 
> node (which may stripe on two or more internal drives) and would fill up 
> 1 Gbit/s to the client.
>
> But a single client may only write at 1000/8/4 = 31 MByte/s since all 
> data will be sent four times, one time to each node.
>
> So my thought was just if it is possible to combine the performance of a 
> server based afr-translator for writes and client-based afr-translator 
> for reads. I guess mounting the same files twice, once in a file system 
> with a server based AFR-translator (write) and once with a client based 
> AFR-translator (read) would get good throughput.

Have you tried chaining AFR volumes?  There's quite a few ways I can 
imagine reducing line saturation if that's the problem.  Here's one:

server1 has an afr of a local volume and a volume from server2

server3 has an afr of a local volume and a volume from server4

client afr of server1's afr and server2's afr

This should allow a single write from a client at 1000/8/2 = 62.5 
MByte/s.  Client writes only twice (to server1 and server3) halving the 
bandwidth.  Server1 and server3 write only once each to server2 and 
server4 respectively and receive the writes from the client, halving 
their bandwidth.

Note: I have no idea how well this will perform in reality.  There may 
be enough lag in glusterfs chaining writes that the gains aren't worth 
it, but I suspect since it is effectively pipelining the writes along 
there won't be too much lag.

--

-Kevan Benson
-A-1 Networks