Sequencing targets during a build.

[Christopher Stranc]

Hello,
  I am trying to optimize a build.  The actual build is divided into 3 makefile fragments corresponding to the layers in our product. 

When make needs to start a job and it has multiple targets that all have their dependencies met how can I influence it to choose the one that will lead to the best build time.

My sequencing issue can be explained with the following example:

--------- Makefile ---------
LAYER_1 :=  1-1-slow 1-2-fast 1-3-fast 1-4-fast
LAYER_2 :=  2-1-slow 2-2-fast 2-3-fast 2-4-fast

COMPONENTS := $(LAYER_1) $(LAYER_2)
#COMPONENTS := 2-1-slow $(LAYER_1) $(LAYER_2)

all: $(COMPONENTS)
        echo "All done."

.PHONY: $(COMPONENTS)

1-1-slow:
1-2-fast:
1-3-fast:
1-4-fast:
2-1-slow: 1-1-slow
2-2-fast: 1-1-slow
2-3-fast: 1-1-slow
2-4-fast:

$(COMPONENTS) :
        echo "Start $@"
        -sleep 1
        echo "Complete $@"

------ Marked up output  (make -j 2)------
A: Start 1-1-slow
A: Start 1-2-fast
B: Complete 1-1-slow
B: Start 1-3-fast
C: Complete 1-2-fast
C: Start 1-4-fast
D: Complete 1-3-fast
D: Start 2-2-fast
E: Complete 1-4-fast
E: Start 2-3-fast
F: Complete 2-2-fast
F: Start 2-4-fast
G: Complete 2-3-fast
G: Start 2-1-slow
H: Complete 2-4-fast
H: Complete 2-1-slow
I: All done.

Now the two slow components determine the length of the build.  1-1-slow ends at time B. 
At time B the following targets could be run: 1-3-fast, 1-4-fast, 2-1-slow, 2-2-fast, 2-3-fast and 2-4-fast.
It started 1-3-fast, delaying Start 2-1-slow until time G.

In my real life example if I could start 2-1-slow all the other targets would complete in parallel to it.  Anything that starts before will just delay 2-1-slow and the end of the build.

I would dearly love a way to "hint" to make to run 2-1-slow as soon as it's dependencies are met, with priority over other targets.

So far I have tried:

1.Sequencing the targets in the COMPONENTS list 2-1-slow first so it might be at the beginning of the list for "what to run next". 

Unfortunately the way the build is structured I cannot actually change when the target is first identified.

2. I have looked at order-only prerequisites, but they do not meet the need.

3. Some people have suggested just crankning up the number of jobs so it will get started earlier but that leads to a "spikey" build process that is hard on our infrastructure and would still take the same time if only I could build the right targets in sequence...

Any help is greatly appreciated.

We are running GNU Make version 3.79.1

Chris

Chris Stranc
Build Architect, Product Development Solutions, Information Services
NORTEL NETWORKS
Phone: (204) 888-1637
ESN:   456-7718        
Email: address@hidden
Mail:  Chris Stranc
       4815 Roblin Blvd.
       Winnipeg, Manitoba, Canada
       R3R 0G2