vpath and created directories (and the directory cache)

[Berend Ozceri]

It appears that make checks for the presence of a directory that's added
to the vpath list at the time the vpath directive is encountered and, if
the directory doesn't exist, omits it from the search path for the
specified pattern. As far as I can tell, this is a problem if
directories that are included in vpath directives are actually created
as a part of the build process, so they may not exist at the time the
Makefile is processed.

Honestly, I encountered this problem not because I do what I sketched
out above, but because I was trying to keep make from start caching a
directory until the directory was built via a sub-make, but alas, as I
found out, the mere act of using a vpath directive causes the directory
contents to be pulled into the cache, causing any newly created files to
be "not found" during the vpath search later on.

I understand the notion of "don't do things behind make's back" and let
me say that I'm contorting make into doing something that it wasn't
really designed for (for building hardware, not software; compiling
Verilog code, figuring out hierarchy dependencies, etc.), but the
behavior of the directory cache is still fairly annoying at times. It
seems like tracking the mtime of directories and at the point where all
hope is lost and make would give up anyway, rescanning directories whose
mtimes have changed and trying one last time would be a good compromise
between not losing the speed benefits of the cache and the
predictability of a truly coherent directory state.

Berend