Re: [Automake-NG] Idea: Inverse sort re-builds by time

[Bob Friesenhahn]

On Mon, 3 Sep 2012, Stefano Lattarini wrote:
> > The maximum possible benefit would be determined by the slowest
> > build product (if it had been scheduled to start last).
> >
> > Thoughts?
> I think that such a feature would be better implemented in GNU make
> rather than in Automake-NG (or any other kind of make pre-processor).
> Have you tried to write to the GNU make list to see their reaction
> and feedback?

No.  Based on past recent history, I don't anticipate that GNU make is 
about to do something innovative like this (maybe a GNU make NG 
could).  But I do anticipate that Automake-NG will introduce 
innovative things.

For my own project I can measure timings and re-order the listings in 
Makefile.am by hand but this is not completely effective because the 
Makefile.am is sub-divided across multiple Makefile includes into 
sub-targets and directories and so there is limited control.

Based on measured timings I can see that some source files should be 
made smaller, and/or moved earlier in the build, to improve 
opportunity for parallelism and help avoid lumbering last few targets 
which continue to compile on a few CPU cores while remaining cores are 
idle.  This does not really help for C++ and for linking though.  C++ 
compilation and linking steps can dominate build times.

In my project, measurements of C source file file compile times vary 
wildly from 0.04 to 4.28 seconds on latest CPUs and it is not always 
clear why some source files take such radically different times than 
others.

Unless there is a sudden change to the properties of silicon, CPUs 
will continue to offer ever more cores, which are not much faster than 
cores in preceding generations, or perhaps are even slower. 
Achieving best (uniform) loading of the CPU cores results in shortest 
build times.  Re-ordering of build targets is an optimization which 
can help with CPU core loading.

Even if Automake-NG does not support intelligent re-ordering, it may 
be able to improve build times by trying to build libraries prior to 
executables (libraries usually take longer to link than executables do 
and are more likely to be gating dependencies) even if the Makefile.am 
suggests a different ordering.

Bob
--
Bob Friesenhahn
address@hidden, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/