scalability issue with 'import'

[Wipf, Stefan]

As we are expanding the use of cfengine in our environment we are
running into the following scalability issue:

A simplified version of our current cfagent.conf:

cfagent.conf:
 import: 
  cf.classes
  cf.variables
  ...

As the cf.* files get larger and larger we would like
to be able to nest the imports:

cf.classes:
 import:
  cf.classes1
  cf.classes2  
  ...

and similarly for other cf.* files

The current implementation of the import directive has the effect of
importing
the files in this sequence:

importing cf.classes
importing cf.variables
importing cf.classes1
importing cf.classes2


This causes us serious headaches as classes and variables defined in the
old
cf.classes and used in cf.variables are now not getting defined until it
is
too late.  Thus everytime we break up a cf.* file into subfiles we have
to
restructure everything.

The behavior that we desire for nested 'import' statements:

importing cf.classes
importing cf.classes1
importing cf.classes2
importing cf.variables

To get this behavior we modified 'parse.c' in the following way:
(instead of appending each import at the end of VIMPORT, we insert all 
 imports found in the file at its position in VIMPORT)


diff parse.c.original parse.c.fix
127c127
< { struct Item *ptr;
---
> { struct Item *ptr, *nextptr
216c216,224
<    ParseFile(filename,sp);
---
>    nextptr = ptr->next;
>    while (VIMPORT != NULL) { VIMPORT = VIMPORT->next; }
>    ParseFile(filename,sp); 
>    if (VIMPORT != NULL)
>       {
>       ptr->next = VIMPORT;
>       while (VIMPORT->next != NULL) { VIMPORT = VIMPORT->next; }
>       VIMPORT->next = nextptr;
>       } 


We hope this is useful for others, maybe not, but the immediate
questions
we have are:

- Are we messing with a basic cfengine design feature that we shouldn't
be
  messing with?

- Will it cause us serious problems in later releases of cfengine?

- Are there changes needed in files other than just parse.c?



-- 
Stefan Wipf
swipf@htc.com