Re: Brittle -includes

[David Deutsch]

Hi Paul,

Thank you for getting back to me (and thank you so much for all your work!).

Race Condition - That was poorly worded on my part. I was thinking more
in terms of how make processes each makefile. I currently put the
includes at the end of each (generated) makefile and was thinking that
maybe if it reads (and tries to execute) the recipe before reading (and
making) the "include" makefile, that could produce issues. The
nondeterministic behavior probably isn't one, but it seems weird to me
that the same generated makefiles work in one case and fail in another.

I've replaced all instances of "-include" with "include" and I'm now
running into problems creating .d files in another part of the setup. I
guess because while it's not throwing a warning, it doesn't write or
include that makefile, now? I've read the manual on "How Makefiles Are
Remade" a few dozen times now, but I'm still not too sure about the
initial case - at what point Makefiles that don't yet exist are written.

I'm using Version 4.2.1. Do you think updating to 4.2.92 would make a
difference?


Another question, if I may: As I wrote earlier, I'm mostly doing this to
extend the pattern rule behavior of make by creating makefiles (and thus
targets with their recipe) for paths that match a certain regular
expression. It seems a bit… excessive to write hundreds, if not
thousands of makefiles and I worry that I'm going to run into
scalability issues at some point (luckily, we live in an era of SSDs, so
it hasn't been too much of a pain so far).

Is there a different direction that you could point me to, here, like
using Guile or some other means to solve this problem? Or would I have
to change core make behavior to make something like this happen? Note
that a complicating factor is that I sometimes have make blueprints that
span several targets (because I sometimes have to not just find a
recipe, but that could itself require additional prerequisites).

Best regards,
David


On 10/16/19 2:18 PM, Paul Smith wrote:
> On Wed, 2019-10-16 at 08:10 +0200, David Deutsch wrote:
>> My problem is that I keep running into situations where an .mk is
>> -include'd as seen above, but make never attempts to... make it,
>> resulting in a 'No rule to make target' error. I suspect there is some
>> kind of race condition going on where it does "see" the include, but
>> missing dependency error is triggered before that can occur. This
>> happens consistently, no matter whether I use -j or not.
> If you don't use -j, then it can't be a race condition.  Make is not
> multithreaded so without -j it should be completely deterministic.  Are
> you saying that _without_ -j you get nondeterministic failures?
>
> You didn't mention what version of GNU make you're using.  Perhaps
> there's a bug in that version?
>
> Also, note that with newer versions of GNU make you don't need to use
> "-include" to avoid warnings on non-existent makefiles.
>
> I recommend you get to a sufficiently-new version of GNU make and
> change all your "-include" to just "include" and see if that helps.
>