singlecopy nirvana

[Eric Sorenson]

A while back, I posted about using 'control: singlecopy = (on)' for
doing hierarchical "most-specific" copies.

    http://lists.gnu.org/archive/html/help-cfengine/2003-05/msg00013.html

I finally got around to fixing the issues I had with the first iteration of
this idea and I wanted to send on a message documenting how it works in hopes
that someone else may find it interesting or useful. This is mirrored in the
wiki at   http://cfwiki.org/cfwiki/index.php/Singlecopy_Nirvana

The problem is a pretty common one: you have a configuration file that
is the same across a class of machines, different between classes, and
maybe has a couple of special-case exceptions for one-off hosts. You
want to manage the configfile in revision control and distribute it via
cfengine, but it's unwieldy to use "copy: myclass.!specialhost::" to
pick the right file out of the repository. What you'd really like is a
way to do "hierarchical" copies: to provide a list of alternatives and
have cfengine pick the most applicable one for a given host. Here's how.

There are three parts to the solution. First, set 'control: singlecopy = (on)'.
This will cause cfengine to treat multiple 'copy:' statements with the
same destination file as a hierarchical copy instead of an error.


 control:

    singlecopy = ( on )
    DefaultCopyType = ( checksum )


Second, create a variable and define it to a different value for each class of
machine that'll have variant files. This is a key step, because it allows you
to use your 'groups:' section to pick the "best" variant, without requiring
'copy: myclass::' style decision trees. I use the generic-sounding $(role),
but as long as it's not a reserved word, it doesn't matter.  Note that this
step is not needed if, as in the example, you use a builtin like ${ostype}
    http://www.cfengine.org/docs/cfengine-Reference.html#singlecopy
But I have not found my config files to vary strictly with ostype, so it's
nice to be able to use arbitrary 'groups:' definitions to expand a variable.

 groups:
    secure = ( robotron sinistar joust )

 control:
   AllowRedefinitionOf = ( role )
   any::    role       = ( nevermatch )
   secure:: role       = ( secure )

( That odd definition for 'any:: role = (nevermatch)' is because if a variable 
is
undefined, cfengine seems to pass it through as a literal string, rather than
empty, so I saw file requests for a file named 'myfile.conf.$(role)' )

Lastly, set up your copy section (and matching files in the repository)
such that there are copy statments for each level of specificity that 
you need to look for.

 control:
    dr = ( /path/to/repository )
    fs = ( cfmaster )

 copy:
      $(dr)/etc/ldap.conf.$(host) server=$(fs) dest=/etc/ldap.conf
      $(dr)/etc/ldap.conf.$(role) server=$(fs) dest=/etc/ldap.conf
      $(dr)/etc/ldap.conf server=$(fs) dest=/etc/ldap.conf

So, using the example 'groups:', if our filesystem looked like

 [eric@cfmaster /path/to/repository/etc]$ ls ldap.conf*
 ldap.conf  ldap.conf.joust  ldap.conf.secure

then robotron and sinistar would get 'ldap.conf.secure', joust would get
'ldap.conf.joust', and everybody else would get 'ldap.conf'.  This lets
you keep a nice clean sparse filesystem in CVS, and you can very easily
tell which file a particular host will pick.

The only downside to this that I found was that cfservd generates a bunch
of error messages every time a client tries to stat a file that doesn't
exist. This happens a lot, because everyone will try to look for a file 
with their hostname extension and almost everyone will look for a .$(role)
extension for their role, most of which will fail. I made the attached
patch to cfservd, which reduced the log volume tremendously (we were over 
700 megabytes a week!) but still makes the error messages available if
you run cfservd in the foreground with '-v'.

I know I bitch a lot about cfengine but this is a very cool feature that
has greatly simplified our config mgmt infrastructure, so I wanted to
write it up and share it... Thanks Mark! :)

--

 - Eric Sorenson - Explosive Networking - http://eric.explosive.net -

cfservd-log.patch
Description: Text document