[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

RE: Editfiles Considered Harmful (was: Re: Complex Editfiles Examples)

From: Wheeler, John
Subject: RE: Editfiles Considered Harmful (was: Re: Complex Editfiles Examples)
Date: Thu, 4 Dec 2003 09:22:33 -0600

Nice argument. I would argue that edit files is not completely useless.
I use it frequently to do token replacement on a pre-seed file from CVS,
although this is probably at the border of your notion of "syntax aware"

This has helped my decision making of managing password files. I was
going down the path of using shell commands to update the passwords file
(useradd), but was unsure of how to actually maintain the entries (shell
changes, password changes, group changes, etc). It became unwieldy to
maintain all the fields for more than a few users.

I think now I'll use something like single copy to push this out.

> -----Original Message-----
> From: Eric Sorenson [mailto:address@hidden
> Sent: Thursday, December 04, 2003 1:48 AM
> To: 
> Subject: Editfiles Considered Harmful (was: Re: Complex Editfiles
> Examples)
> On Thu, 4 Dec 2003, Jamie Wilkinson wrote:
> > This one time, at band camp, Russell Adams wrote:
> > >Egad man!! I'm impressed. ;]
> > >
> > >I setup a similar construct last night to edit Gentoo's
> > >/etc/make.conf, but its not nearly as complex.
> >
> > My best editfiles is the one for /etc/ssh/sshd_config, most of my
> > configuration file edits are based on that.
> It really terrifies me when I see stuff like this.
> Not that your editfiles won't work (necessarily); it's just scary to
> that
> as a general practice, people let tools generate syntax which have no
> themselves whether the syntax is valid.
> This is what I meant the other day by std_editfiles_rant.h, but I
> would
> like to know if I'm completely off-base on this so I'll write a bit
> where
> I'm coming from.
> In my first cfengine site install, I never touched the editfiles
> functionality
> because I simply didn't need it. I was mostly shipping binaries around
> keep
> a consistent gnu-tools environment on solaris in a production ISP
> environment
> that eschewed NFS and NIS.
> For the password file distribution problem, I wrote a bit of a perl
> frontend to
> build up the appropriate passwd/shadow files and place them in the
> repository
> for clients to pick up via 'copy:' when they updated.. (this lives on
> all
> its cobwebby glory at ) This
> worked pretty well and I didn't get bit too badly by any bugs -- even
> cfengine 1.5.x!
> On to my current site, where I wasn't directly involved in the
> cfengine deployment. It's an internal corporate network and we were
> butting up
> against NIS limits but had NFS available everwhere, so the main
> for
> cfengine was to build up passwd, shadow, and group from fragments over
> NFS.
> The implementor used editfiles for this task, and we started seeing
> edge
> cases like zero-length shadow files, corrupted entries, etc. Some of
> were
> parse bugs or things not working as expected, some were implementation
> problems
> in our configs -- the blame for which I still ultimately pin on
> strange set of features and commands, as has been amply discussed on
> list.
> But as I was trying to hone an alternative to this for our new
> cfengine2-based setup, I had to pin down exactly what bugged me about
> editfiles problem in order to convince the rest of the team that we
> needed to eliminate it from our vocabulary as an operating pragma, and
> instead
> copy whole files from a version-controlled repository.  This was the
> Thing to Do, I felt it in my gut, but it wasn't easy to articulate.  I
> think
> I've got it pretty well sussed, and it turns out there's really two
> issues: one
> has to do with systemic behavior and the other is about management
> complexity.
> First, let's look at a typical goal of an editfiles task. Usually
> updating a system configuration file from its default state, in order
> bring
> the behavior of a daemon/subsystem in line with policy. After the
> time it
> runs, the editfiles stanza ought to be a no-op; this is the stated
> of
> commands like AppendIfNoLineMatching. The problem with this approach
> that
> the product of the original file state run through editfiles actions
> cannot be guaranteed to be equivalent to the desired end state, if
> no semantic rigor imposed on the actions performed. This is the idea
> behind
> Alva Couch's "conduits", and in no sense are the regexp description
> editfiles commands a conduit for any of the things editfiles's
> editing.
> Put simply, there's nothing inherent in editfiles which prevents you
> accidentally scribbling nonsense all over a live file on the target
> machine.
> This is incredibly dangerous for anything of significance! I've seen
> problems
> where 'AppendIfNoLinesMatching' didn't match when it should have,
> broken-record repetition. Substring matches for 'LocateLineMatching'
> match too early or not at all (usually depending on whether or not you
> use ^ and $ anchors correctly).  Worse than all of this, the baseline
> behavior of the subsystem in question may have changed and your edits
> do not produce the intended consequences -- even though editfiles is
> exactly what you've told it to do!
> Contrast this with editing or generating a config file, testing that
> on a
> non-production system, and then "promoting" it into production by
> it
> in the repository. Unlike a series of adds, deletes and edits to the
> contents
> of the file, I have a high degree of certainty what the target system
> look
> like (and by extension, how it will behave) at the end of the cfengine
> run.
> In a related sense, the second problem shows iteself when someone else
> tries to modify a modification you've made, or when the policy
> necessitating a change in the system's behavior. Now you have to
> from the configuration files as they exist on the production systems,
> into the guts of your editfiles cleverness (which by now has descended
> mere unreadability) and reverse-engineer the adds, deletes, and edits,
> in order to add a new directive to the top of the file.
> Why not just edit the file with an editor and check it in to the
> repository?
> Let cfengine copy the new version over and restart the deamon.  I'd
> that this is the end result that you want -- a new version of
> on
> all machines of a particular class (or whatever), so why not just do
> Obviously there's been a lot of work put into making the editfiles
> set very rich, but it's never going to be able to handle all the cases
> we run into without becoming (even more) hopelessly tangled,
> regardless, it'll never be as smart as a human with a text editor.
> The final blow to the editfiles methodology should be clear in the
> not-so-hypothetical situation where someone asks you to show them what
> changed
> between last week and this week in the ssh subsystems of your
> This
> is, and should be, a text diff between two revisions of the
> file.
> It should NOT be a text diff between two revisions of your editfiles
> section,
> where you have to pull out the whole stanza from each version *and the
> original
> configuration file* and compare the results of applying each stanza to
> copy
> of the config... That's crazy-talk, absurd on its face.
> Use CVS. Make a directory tree that looks like a filesystem, and
> each
> leaf directory with the files you want to manage. Either make separate
> trees
> per host-class or use class-keyed filename extensions in a unified
> it
> really doesn't matter. Use copy: with checksums, not date-stamps, so
> machines
> which get installed after your latest config change still get the
> authoritative
> version.  Use a define=className in your copy directive to specify
> triggers for
> restarting the affected service daemon (remembering to pre-declare
> className
> with AddInstallables in your control: section).
> Voila, all your editfiles problems just disappeared.
> Seriously though, I would like to hear any criticism or feedback on
> above -- if there's something that just doesn't fit into this model,
> maybe some loon^H^H^H^Hfine person who feels as strongly Pro-editfiles
> I do
> Anti-, I'd welcome the dialog.
> --
>     Eric Sorenson - EXPLOSIVE Networking -
> _______________________________________________
> Help-cfengine mailing list
> address@hidden

reply via email to

[Prev in Thread] Current Thread [Next in Thread]