Re: ctime vs checksum in copies

[Frank Ranner]

Paul Krizak wrote:
> I've got a question about what you guys think the best solution is in 
> this situation.
>
> I've got some files that must be customized on a per-host basis, but are 
> based on a common "root" file.  My goal is to have the following system:
>
> * When the system is first installed, or if the file on the system is 
> wildly different than the master, a copy: statement pulls down a fresh 
> copy of the file, which an edifiles: stanza corrects for that specific 
> host.
>
> * Later, if the master file changes, the clients should "know" to pull 
> down a new copy of the file, and then perform the same editfiles: stanza 
> again.
>
> Using "checksum" for the copy statement doesn't really work, since once 
> you make a change to the file using editfiles:, the checksums don't 
> match and thus the file gets copied every time.  On the upside, you're 
> guaranteed to always have the correct version of the file on the target 
> system.
>
> Using "ctime" for the copy statement *sounds* good, as the copy only 
> happens when the "master" file updates on the server.  Where this falls 
> flat, however, is on freshly installed systems, where the ctime of the 
> *incorrect*, freshly-installed file is *later* than the ctime of the 
> "master" file.  In this situation, the "master" file is not copied (and 
> I wouldn't expect it to).
>
> I've got several cases in my (very lengthy) cfengine config where I run 
> into this type of issue.  For small files, I can put the entire contents 
> of the file into an editfiles: block and avoid using copy: at all.  This 
> doesn't work for large files, however, as I don't want a cf.* file to 
> have a 5,000-line config file in it with Append statements.
>
> For large files, I use checksum copies and just accept the fact that 
> they will get copied every time cfagent runs.
>
> I'm looking to improve convergence in my config files...anybody else 
> have clever workarounds for this problem?
cfengine is a great general purpose tool, but is very clunky for certain 
purposes. The example you give is one which is possible to implement as 
a pure cfengine solution, but which can be made easier by bringing in 
more specific 'helper' apps.

To create a 'target' file based on a 'source' file, along with some 
parameters and processing rule is territory well covered by 'make'. 
Simply copy the template and the Makefile using checksum, and invoke a 
make in shellcommands. A single Makefile can have the rules for creating 
any number of config files and can run make in any number of other 
direcories. Running 'make' on up-to-date files is generally quick, but 
triggering make using a class may be more efficient. Perhaps something 
like 'main.(make_needed|Hr02)::'. When I use make in cron jobs I usually 
pipe the output into grep -v to filter out 'makefile is up to date' 
messages, so I only get mail when the makefile triggers.

To perform the transformation from template to production file you can 
use awk, sed, m4. m4 is a particulary good choice as it allows most of 
the decision making to be imbedded in the template, steered by variables 
that can be defined on the command line. m4 allows simple substituion, 
conditional text (ifdef and ifelse), file inclusion (include) and 
complex macro expansion.

Regards
Frank Ranner