Re: Hierarchical make: How to avoid unnecessarily going into certain directories?

[Peng Yu]

On Sat, Jul 31, 2010 at 11:05 PM, Philip Guenther <address@hidden> wrote:
> On Fri, Jul 30, 2010 at 11:21 PM, Peng Yu <address@hidden> wrote:
>> My questions is how to more efficiently run make in hierarchy of
>> directories. Let me give the following example to demonstrate what I
>> mean.
>
> The short answer is that you should read the paper "Recursive Make
> Considered Harmful".  It can be found on the web at:
>        http://miller.emu.id.au/pmiller/books/rmch/

This paper describes a simple problem in a rather complex way. If I
understand it correctly, the main point is that if the dependencies
are separated in multiple Makefiles, and none of the Makefile has all
the information. Then it may not correctly make the targets. The paper
gives some fixes, but none of them are satisfactory. So the paper
suggests to abandon recursive make altogether.

But this problem is not applicable to the example that I gave in my
previous email. But my previous example is too simple to fully explain
my hierarchical Make rules. Let me give a more complex examples.
Suppose that I have a hierarchy of directories,


a/b/c/
a/b/d/
a/e/
f/

My hierarchical Make rules are:
1. A directory depends something not in the current level but not
anything in the current level.
     Eg. a/b/c/ may depends on something in a/e/ but not anything in a/b/d/.
     The rules in the Makefile in a/b/ is
     .PHONY: all
     .PHONY: a/b/c/.dir
     all: a/b/c/.dir
     a/b/c/.dir:
        $(MAKE) -C $(dir $@)

2. A directory depends something not in the current level and
something in the current level.
     Eg. a/b/c/ depends on something in in a/e/ and something in a/b/d/.
     The rules in the Makefile in a/b/ is
     .PHONY: all
     .PHONY: a/b/c/.dir
     all: a/b/c/.dir
     a/b/c/.dir: a/b/d/.dir
        $(MAKE) -C $(dir $@)

3. A directory doesn't depend on anything outside.
     Eg. a/b/c/ doesn't depend on anything outside a/b/c/
     The rules in the Makefile in a/b/ is the same that in case 1.

4. If anything daughter directory depends on a third directory in the
current level, then the parent directory also depends the third
directory.
     Eg. a/b/c/ depends on something in a/e/
     The rules in the Makefile in a/ is
     PHONY: all
     all: a/b/.dir
     a/b/.dir: a/e/.dir
        $(MAKE) -C $(dir $@)

     Note that if something in a/b/ depends on something outside a/,
the Makefile may include
     PHONY: a/b/.dir

5. The last target of any directory is the file .dir, so that it can
be used to tell which directory is newer in the same level.
   E.g. The Makefile in a/b/c/ should include the following lines
(.dir should depend on everything that actually should be made in
a/b/c/)
   PHONY: all
   all: .dir

   .dir: something1
       do something

   .dir: something1
       do another thing

6. The files are organized so that there is no cyclic dependence.
    E.g. a/b/c/ depends on something in a/b/d, then nothing in a/b/d
depends on anything a/b/c/, because I can always restructure the
directories to eliminate such cyclic dependencies.

Once a hierarchy of directories follows the above 6 rules, I think
that there will be no errors with recursive make except that Make may
going into some directory only find out that nothing need to be done
(as shown in the example in OP).

Note that I don't use any VPATH in the Makefiles, rather I spell out
all the relative path.

Please let me know if the above 6 rules are not sufficient to make
sure Make doesn't miss any dependence as I may miss something. But
suppose the 6 rules are complete, then there is no need to avoid
recursive make


> Fixing your build system to not use recursive make solves the "big
> picture" problem.

I need to to flexibly work in any directory, i.e. I can run make in
any sub-directory. I'm not sure how to achieve this without recursive
make. Note that, I don't quite understand how section 6 in "Recursive
Make Considered Harmful" could be applicable to my cases. In my case,
I only have a single repository (not sand-box and master source). If
you think that VPATH can solve my problem, would you please show me
how to use VPATH to remove recursive make in the example in OP?

-- 
Regards,
Peng