Re: [Gluster-devel] Early review/feedback for glusterfs 4.0 plan

[Anand Avati]

On Wed, Dec 11, 2013 at 11:19 PM, James <address@hidden> wrote:

Thanks... I just read your 4.0 doc. I have a few comments:

1) Someone talked to me a bit about the sub bricks idea. I haven't
decided how awesome or not this would be, but I'm feeling positive
about the change. See 2b for a related comment. I'm keen to see
Gluster gaining an understanding of "tiered" storage.

2) I wasn't sure if I understood certain parts correctly. Depending on
if I did or didn't, I think there may or may not be some significant
problems with the suggested model for building higher level management
tools (eg: puppet-gluster). I would have to know more to know for
sure. For one, I got the feeling that some commands needed to be
executed one after another in a certain order...

2b) If someone can help with the algorithms I mentioned in:
https://lists.gnu.org/archive/html/gluster-devel/2013-11/msg00135.html
  I think I'll be able to provide a convincing case that Gluster
management isn't as bad as you might be alluding to in your 4.0
comments. I think a lot of the GlusterFS core team are allergic to
Puppet. I think this is quite normal, because Gluster core is super
low level C, where as Puppet (a mostly declarative language) is super
high level and on the opposite side of this spectrum. However, I think
most people are discounting the importance of having something like
this. The future is _all_ configuration management. The early adopters
are almost mostly there.

That's probably not entirely the case. I don't think there is anybody who disagrees on the need/importance of puppet and puppet-gluster. I also see how it becomes almost necessary on a 10k node cluster, and why it is important to make gluster integrate nicely with the puppet ecosystem.

At the same time, the reason why puppet-gluster module is having to solve complex issues is because gluster management isn't doing the right job. The algorithms you are working for puppet-gluster are not puppet specific and their right "location" is in the core of gluster. It doesn't feel right to see such "dense" logic in the puppet layer while that intelligence is needed pretty much for every one.

I'm hoping 4.0 will make the gluster-puppet module simple and elegant, and not "unnecessary".

3) Between early Gluster x.y and 3.x I had to completely change
Puppet-Gluster to support the new management style. The current
Puppet-Gluster stuff is easily one of the most complicated Puppet
modules that exists _anywhere_! This is partly due to the fact that
while Gluster might be easy to manage (and even logical) for a human,
it is _not_ from an automation point of view.

That's precisely my point as well. The complexity in the puppet module is for a problem which has to be fixed somewhere. And I think the right place for that extra logic to reside is within the gluster management layer. Puppet module should not need to know how gluster is distributing and replicating data - it is too low level/internal knowledge to be exposed up to a puppet-like layer.

 

In any case, I've built
it, but there are certainly ways that Gluster internal could make it
easier (thank goodness for --xml anyways ;))

If Gluster 4.0 radically breaks all the work I've done for current
Puppet-Gluster, I really won't have any desire to rewrite all of this
for a third time without a large pay cheque from RedHat. On the plus
side, this could be a good way to get rid of me ;) Either way, whether
you consult with me, or you consult with a different configuration
management expert, do the smart thing: find out if the management
style you're proposing is compatible with configuration management. If
it will work elegantly with Puppet, it will work for Chef, etc...
Usually when something "fits" really well with well designed Puppet,
it's a sign that the thing was designed _extremely well_.

As an example of this, the FreeIPA team (RedHat) did a kick ass job
putting together the FreeIPA management interfaces, and as a result,
it is extremely pleasant to use natively, and also with Puppet.
Example: https://github.com/purpleidea/puppet-ipa

Will check that out, thanks for the pointer.

Thanks!

Avati

4) Looking forward to 4.0 ! I hope I have a chance to see the going
ons very early, while there's still a chance for me to have a positive
influence on the external interfaces.

Happy hacking
Hope my comments are helpful/useful,

James

PS: I know that some at RedHat and Gluster core will probably think
this is insane, but think about it: In the future (maybe that's now)
software will ship natively with some sort of (probably mostly
declarative) "front end" as the recommended UI for management. This
means it makes sense to have some interfaces in native code, and some
as higher level abstractions around your native interfaces. Think of
the analogy of Gluster in C, but perhaps glusterd in higher level
python. Now realize that the "next level" might be something like
Puppet. I think it's already possible, when you include some of my
Puppet hacks, Example: https://github.com/purpleidea/puppet-pushing