Semi-Parallelizing

[EXT-Pennington, Dale K]

Folks,

We have a relatively complicated system we use gmake to build. But it is
now taking long enough to build that we want to investigate taking
advantage of the -j option to speed it up. But I had a few items I just
wanted to make sure of, and one real question on best way to something.

In general we have many subdirectories in our src tree. Some are used to
build libraries, the others are used to build the final applications to
that link with the libraries we built.

We have a top level make that basicallly invokes make on each of the src
dirs. Right now, we take advantage of the serial nature of make to make
the library directories before the bin directories, but that can be
handled by just adding dependencies to show the bin directories depend
on the lib directories (I think). The other issue at this level is when
making the actual libraries, we do not want to do the actual library
compiles in parallel due to ar issues. So, if I explicitly pass -j 1 in
the command line to the submake invocation, will that override and
passed down -j flag from the master make ? 

Example (Leaving out the detail of creating dummy targets from the dir
names to simplify):
LIB_DIRS = a b
SRC_DIRS = c d

$(SRC_DIRS) : $(LIB_DIRS)

$(LIB_DIRS) :
        gmake -C $@ -j 1

$(SRC_DIRS) :
        gmake -C $@


Would this do what I described above ? 

At the src dir level, we actually have 3 standard make targets, all,
which is a strait compile, clean, which removed all compilation
products, and CM, which is supposed to force a total rebuild of that
directory, then remove the intermeditate compilation products. In
general they tend to look like

all : $(EXE_NAME) <possibly more>

$(EXE_NAME) : <dependences>
        <commands to make EXE_NAME>

clean_objs :
        -rm -f $(OBJS) $(DEPFILES)

clean : clean_objs
        -rm -f $(EXE_NAME) ...

CM : clean all clean_objs

The real problem child in this case is CM. As long as -j is not
specified (or is a 1), then the order that gmake issues build requests
make it work (I am not sure gmake guarantees this, so this may not be
the safest design anyways). But as soon as we use the -j flag, the CM
option is clearly broken. So far I have not figured out a way to do this
without basically duplicating a lot of the targets actions. In
particular I would prefer not to have to replicate the $(EXE_NAME)
stuff, and thus far I have avoided recursively invoking make (although I
guess that might end up being the best option, something like)

CM :
        -rm -f $(EXE_NAME) $(OBJS) $(DEPFILES)
        gmake all
        -rm -f $(OBJS) $(DEP_FILES)

Am I missing something that would work better.

Thanks

Dale Pennington