Re: [Gluster-devel] Adding nodes

[Kevan Benson]

Marcus Herou wrote:
> Thanks for the answer Kevan. I've read quite a lot about GlusterFS but
> haven't got the chance yet to test it. (Board dp if I can buy all that
> hardware :)
>
> However the conf was ripped from
> http://www.gluster.org/docs/index.php/Unify_over_AFR but stripped down to
> three instead of six nodes.
>
> Please show me the error it would be more helpful than a 1000 lines of
> reading.

Specifically, you can't have a volume (e.g. brick2 in your config) be a 
sub-volume of two different AFRs.  The solution is to have each server 
export two volumes, such as brick2-1 and brick2-2, that way each volume 
can be a member in it's own AFR.

Extending this to three servers and three AFRs, you get:
AFR1 { brick1-2, brick2-1 }
AFR2 { brick2-2, brick3-1 }
AFR3 { brick3-2, brick1-1 }

This makes adding a new server (brick4) a bit more complicated, as if 
you want everything to line up pretty like above you would want to 
copy/move all brick1-1's data to brick4-1, and use the following AFR 
definitions:
AFR1 { brick1-2, brick2-1 }
AFR2 { brick2-2, brick3-1 }
AFR3 { brick3-2, brick4-1 }
AFR4 { brick4-2, brick1-1 }


> Thanks for the headsup of find... So adding a server which needs TB of data
> would not be something you would like to execute everyday I guess... Can you
> prefill it with yesterdays possibly "stale" data and then execute the same
> find so only new/updated files get's transferred ? I would like to have it
> as warm as possible before putting it online if you know what I mean.

There were some recent threads on this, and problems someone was 
encountering with pre-populating data, but that might have been with 
deploying the initial data set (where the xattrs holding AFR file 
versions weren't set).

I assume you could rsync the active file set from another node of the 
AFR to the new server with -X to preserve xattrs and go from there, but 
it's been a while since I looked in detail at the xattrs used to keep 
AFR state.  I'm sure someone else will jump in with more info on this, 
as it seems to be an active (or at least frequent) topic on the list.

--

-Kevan Benson
-A-1 Networks