Re: [Cfengine-develop] my own branch of the tree

[Luke A. Kanies]

Hi Mark,

I apparently managed to delete your message while purging spam.  Skaar let
me know about it and helpfully sent me a copy.

On Tue, 13 Jan 2004, skaar wrote:

> ----- Forwarded message from address@hidden -----
>
> Date: Sat, 10 Jan 2004 08:40:32 +0100 (MET)
> From: address@hidden
> Subject: Re: [Cfengine-develop] my own branch of the tree
> To: address@hidden
> Cc: address@hidden
>
>
> Luke, your timing is strange as I just sent you a message to say that
> I have included your patch. I welcome help in these matters but I
> am still very scared that folks will succumb to the "easy way" of
> scripting "solutions" to problems rather than finding the reproducible,
> general, convergent way. Cfengine has to be state of the art -- oops I mean
> science.

I agree that the timing is strange.  I haven't looked into who sent what
first, but I have had my email sitting in my postponed messages box for
weeks, trying to decide the right course of action.

As to concerns about people finding the easy way...  On the one hand, that
is certainly my goal:  It is difficult and often impossible to do
moderately complex tasks in cfengine right now, and I want it to be
easier.  However, I think your real concern is cfengine morphing into
"easy" rather than "right".  I don't remember the author, but isn't the
quote "Things should be as simple as they can be, but no simpler"?
Cfengine is not as simple, nor as powerful, as it can be, and I would like
to fix that.  I do agree that it should not compromise its founding
principles to become easier to use, but I think it has a long way to go
before that becomes a problem.

As to the reason why I am constantly writing scripts that cfengine calls,
it is because cfengine cannot do what I require.  90% of that would be
resolved by some sort of built-in list variable and an iteration operator.

> After LISA I was more worried than ever that people are not only not
> so worried about this, but that some people are even going backwards
> and writing brand new tools that look like ftp.

Some people have different needs.  I manage a network of four machines at
home, and I certainly don't manage that with the same techniques I use at
work.

> You have earlier criticised me for being against abstraction. I do not
> believe that this is correct. I am against ambiguity. Cfengine has its
> problems and I would love help in working them out, at the same time
> as moving forward with the anomaly stuff, but I am scared of what I said
> above -- that in its unbridled haste, the wheels of progress will
> maim the key principles that make cfengine safe today.
> e.g. the whole discussion about locking and dual passes; peoples'
> insistence on writing modules for every little thing instead of
> developing the convergent mechanisms...

I don't know what your reasoning is for how cfengine turned out, I can
only see the product, and cfengine itself is somewhat pathologically
opposed to abstraction.  Methods are an attempt to build some abstraction,
I believe, but without iteration they are insufficient.

In nearly all cases, the modules I write are to provide some facsimile of
iteration, and sometimes to provide an attempt at higher-level data
structures.  For instance, I recently wrote a module to install packages,
as I described on the cfengine list.  To do this, I had to collect a list
of packages to install, along with the specific revision to install, and
then I had to execute the code necessary to do that work.

Well, cfengine does not have a list type, so I had to collect the list in
a string.  Cfengine does not support a hash type, either, so I had to name
the packages like <package>--<revision>, e.g., cfengine--2.1.0p1.  And
because cfengine does not have any kind of iteration, I had to pass that
list to an external module to get turned into the effectively iterative
process.  And lastly, because cfengine does not have anything resembling
'eval', I had to set up a confusing system whereby the file gets generated
on the first cfengine run, and the next run it gets imported because it
exists.  This really, really confuses most people I work with, but there's
no other good way to do this within cfengine right now.

As to why you don't ever run into these problems, I don't know.  I am
astounded that cfengine meets all of your system management needs, and if
they do, you are the only person with I've spoken for whom that is true.
Everyone else is stepping outside of cfengine for very significant
operations, either generating cfengine code (something Alva recommended as
the "best" solution a long time ago) or just avoiding cfengine entirely
for swaths of their system management.

I think cfengine is a great tool with a great pedigree, but taking it to
the next level is going to require some significant changes.  Now, I
believe it is possible to enact those changes while maintaining the
original goals of cfengine.  If you don't trust other people to interpret
that goal, then that's your prerogative, but we who are interested in
developing cfengine obviously do not have quite the same goals as you do.

I _personally_ find cfengine barely usable without iteration and lists.
Whether others agree, I don't know.  I _personally_ find cfengine's parser
to be the source of many, many problems, and much confusion.  Again, I
have no idea if others find that to be true.  But my goal is to spend the
smallest amount of time on cfengine fixing the biggest problems.  I don't
think I'll be able to easily add a list type, and I don't think I'll be
able to easily fix the parser (BTW, you should look at the parser for
bash; it's an excellent example).  However, I do think I can solve some of
the glaring problems, and bring some modicum of consistency to cfengine,
and that is certainly horribly lacking.

> The development team was put together from people whom I trust
> to do good work, but it has been very quiet. The Savannah web pages
> have been a complete waste of time, as I suspected. I don't even
> understand how to maintain them anymore. They have become a burden
> rather than a help. At times I will be too busy to even think about
> cfengine -- but I have a few weeks now. Let's talk.

I am not surprised that the development team have not accomplished
anything; they don't have CVS (or any other version tool) access, which
means that they have to implement their own synchronization mechanisms.
You have also expressed more fear of mistakes than interest in
advancement.  I think the fundamental disconnect is that you find cfengine
essentially satisfactory, whereas others (at least me) think there are
areas where it needs specific improvement.

If you are the gatekeeper to changes in cfengine, and you don't think it
needs to be changed, then the development group is naturally not going to
get much done.  At some point you have to learn which developers you trust
to keep your vision, and then let those developers into your inner circle.
You are apparently too busy to continue advancing cfengine very much, and
that's fine; you've been essentially the sole developer for 10 years.  But
you need to be able to at least pass the workload off on someone else.

I have gotten the distinct impression that you don't trust my work,
apparently because I have written in perl.  I actually think I have a
pretty good idea of the cfengine vision, and I think I've done well at
keeping that vision even in the modules I've written.  I just have the
need for abstraction, I want a quality parser, and maybe a few other
things.

Also, I plan on offering commercial support this year, which means I need
to be able to fix bugs that my customers submit to me without waiting for
a lengthy approval from you.  For this reason alone I will have to
maintain my own branch, but it would be great if that branch were
essentially in sync with the main branch.

Luke

-- 
Once...in the wilds of Afghanistan, I lost my corkscrew, and we were
forced to live on nothing but food and water for days.  -- W. C. Fields