[Fwd: Re: Getting Group "Contents"]

[Ed Brown]

I never saw this come out on the old list last Thursday, so retrying to
the new list...  It's part reply, part suggestions for cfengine 3.

I'd add that over the last 5 years, I've seen a lot of otherwise good
programmers and admins struggle with embracing/adopting cfengine because
of difficulty understanding or trusting what it's going to do or what it
is doing.  I think that improved simple, concise, clear reporting and
querying abilities could really take cfengine 3 to another level of
adoption.

-Ed


-------- Forwarded Message --------
From: Ed Brown <ebrown@lanl.gov>
To: Harrison, Bruce (CXO) <Bruce.Harrison@hp.com>
Cc: help-cfengine@cfengine.org, Mark Burgess <Mark.Burgess@iu.hio.no>
Subject: Re: Getting Group "Contents"
Date: Thu, 16 Mar 2006 20:01:53 -0700
On Wed, 2006-03-15 at 10:53 -0700, Harrison, Bruce (CXO) wrote:
>     I need to be able to get the "contents" or value if you will of a
> cfengine group from a command line and/or perl script.  
...
> I am going to write a perl sub to do this, but thought I would check
> here first to see if this functionality already exists somewhere.  

It's hugely useful to be able to do this, unfortunately the
functionality is not native to cfengine (maybe for cfengine3 ??).  The
problem is not easy to solve in a general way, and from cfengine's point
of view, it just isn't intended that any node can know about classes
(groups) defined on any other node, it just doesn't work that way.  

If you undertake your own perl solution, there's lots of wrinkles: do
you only parse one file (groups.cf, perhaps) or everything in inputs/?
What if all files, i.e. all groups definitions, aren't on every machine?
What if group membership is conditional?  What about the variety of
expressions allowed in groups definitions, including groups within other
groups?  So you might end up restricting, to some degree at least, what
class information is available like this, that is, outside of a running
instance of cfagent, and possibly the range of syntactical expressions
that would otherwise be allowed in the groups definitions.

But once you accept some limitations (I wonder if a recognized
distinction wouldn't be useful here: a new name denoting user-defined
'static', or constant, groups/classes, versus all others; then maybe
cfengine itself would be able to reasonably provide some info, from the
inputs it has available anyway), anyway, once you accept some
limitations, many possibilities emerge and cfengine classes suddenly
become even more powerful as a way to represent your network.

We've been doing this for a few years now, with lots of site-specific
assumptions built into the process, so I'm not sure how generally useful
my scripts would be.  A cfengine module on the servers parse groups.cf
files belonging to various teams.  A CSV-type file is created:
hostname,class1,class2..., and distributed to all nodes, along with a
utility for querying the data.  With some limitations, the utility can
take compound/complex expressions like you might use to qualify a
cfagent action, so it can be used to check in advance what systems a
given action might apply to, (showme 'WebServers.!DevelopmentNet.(TeamA|
TeamB)'), or used in various other scripts and applications, like
cronjobs or backup routines, Nagios and iptables configs...

It's one way I'm able to make cfengine configuration information more
visible  (and useful) to all admins.  This was something Luke Kanies
mentioned at the recent LISA conference as a worthwhile objective toward
helping others _understand_ configuration management, which can seem
opaque, even to those involved in creating the configurations.

Mark, would this kind of native capability in cfengine3 be a
possibility, or is it just not within the scope or design philosophy
or architecture for cfengine development?    

I'd also love to see other kinds of querying capabilities, with clear
easy-to-grok output (I know lots of this info is available now, but it
can involve piping and grepping and sorting through lots of arcane (to
the uninitiated) "extra" information.), something like these examples:

cfshow inputfiles  
(would be like:  cfagent -qvp |grep "Looking for an input file" | \
 awk '{print $6}')
cfshow editfiles
(would act like a full dry-run (not a '--just editfiles', which can be
different in results) and only show the editfiles information, not all
the preliminaries, and context switches, etc)

Similarly, additional modes for describing what cfengine did, for example:

cfagent -q --summary  (because '-I' can be too much, or too hard to
read)
cfengine: webserver.cf: edited /etc/httpd/conf/httpd.conf
cfengine: site.cf: copied /source/bar to /etc/bar
cfengine: bind.cf: sent signal HUP to PID 123
...

This ability to provide clear "views" into the effective configurations
(that might be spread over several files and be otherwise difficult to
grasp) would make cfengine far more _friendly_ to new and experienced
users alike. 

Sorry to get off the topic of your question, it is a good one though.

thanks,
Ed