Re: [Gluster-devel] clustered afr

[Anand Babu]

Tibor Veres writes:
> volume afr
>     type cluster/afr
>     subvolumes b1 b2 b3
>     option replicate *:2
>     option scheduler rr
>     option rr.limits.min-free-disk 512MB
>     option rr.refresh-interval 10
> end-volume
> This setup sort of works, except that i saw files created only on
> bricks1 and 2, brick3 got only the directories and symlinks created on
> it. After killing the brick2 glusterfsd, the filesystem stayed up,
> which is promising, but still no files are created on brick3.

Scheduler is an option of unify translator. As of now you are using
plain AFR across three bricks with 2 copies. To achieve what you want,
you can try like this. Create 3 AFR volumes across 3 bricks in pairs 
of two like this: 1-2, 2-3 and 3-1. Then unify all the three volumes
using rr scheduler. 
Please refer to this example
http://www.gluster.org/docs/index.php/GlusterFS_User_Guide#AFR_Example_in_Clustered_Mode

Above example should work for you. You can also create simple pairs
like 1-2, 3-4, 5-6. 

> is this setup supposed to work? can i get comparable functionality set
> up with current glusterfs? preferably in a way that can be extended to
> 5 nodes, withstanding 2 going down. is there any plan for some
> raid6-like functionality, or this would kill performance alltogether?

Yes GlusterFS can scale to large number of nodes with high availability.

I am not in favor of implementing RAID6 translator. Because
* Striping every byte range across the RAID volumes will result in
  serious disk contention.
* Parity calculation on client side is very expensive and slow.
* Recovery from corruption will some what scary.

For reliability, it is smarter to just use AFR and Unify in
combination.

--
Anand Babu 
GPG Key ID: 0x62E15A31
Blog [http://ab.freeshell.org]              
The GNU Operating System [http://www.gnu.org]