Re: full VPATH search followed even if prerequisites found in the first entry

[Philip Guenther]

On Wed, Nov 11, 2009 at 9:42 AM, Mark Galeck (CW) <address@hidden> wrote:
> The GNU make manual says:  “ In the VPATH variable, directory names are
> separated by colons or blanks. The order in which directories are listed is
> the order followed by make in its search.”
>
> This does not work for me:
>
>  I have a target that depends on several hundred prerequisites, and a VPATH
> that is several hundred directories long.  However, all the prerequisites
> for that target are found in the current directory “.”, which  is VPATH
> first entry.
>
> To make the target, takes half a minute, where make is finding and checking
> those prerequisites.  I can’t wait that long.  But, if I cut off VPATH so
> that it only lists “.”, it is immediate.  Apparently, make is checking those
> further directories in VPATH, even though all the prerequisites are found.

Try doing a make with the -d option and pay close attention to the
long list of patterns that make is applying to find the prerequisites
for all the files.  For example, with a trivial Makefile, there are
107 possible prerequisite files for the Makefile itself.  Each of
those must be checked for in each of the VPATH directories.  The list
for other files may be longer or shorter depending on its suffix, but
it has to check for them in suffix order, not VPATH directory order.

(Does the Windows version of GNU make not have a directory cache?  If
so, your pain is explained by make having to actually go to the kernel
for each lookup instead of only reading each directory once.)

Removing unnecessary pattern rules can help quite a bit in these
situations.  For example, if you don't use RCS, SCCS, or Cweb, then
canceling the pattern rules for them by adding these lines:

%.c: %.w %.ch
%:: %,v
%:: RCS/%,v
%:: RCS/%
%:: s.%
%:: SCCS/s.%

reduces the number of possible prerequisites for the Makefile from 107
to 17 and reduces it for other files by significant factor.  If you
have a lot of included makefiles and none of them have real rules for
being generated (*.d files don't count, as they can be intentionally
generated as a side-effect), then explicitly marking them as having no
prerequisites with something like:

${MAKEFILE_LIST}: ;

can save make still more effort.


In other words, you're having a performance problem, so treat it like
one by profiling or tracing the process's operations to see what's
taking so long, then figure out how to make the more inefficient steps
more efficient.


Philip Guenther