Re: new feature idea: ingesting processed rulesets

[Paul Smith]

On Sun, 2023-06-11 at 12:29 +0000, Zoltán Turányi wrote:
> My problem is that contrary to the make wisdom of writing a single
> Makefile (to which I agree) most projects are still divided into
> parts with separate build definitions. One can debate if this is good
> or bad - for me it is a requirement to handle this case efficiently.
> (If you do not agree there is no point in discussing further.)

I wouldn't say "requirement", especially in conjunction with
"efficiently".  If people want "more efficient" they may have to be
willing to pay the up-front cost of changing their makefiles and I
think that's a defensible position.

But you are basically saying what I said in my previous email: the only
way it makes sense to try to implement something like this in GNU Make,
is if it could be done in such a way that people DON'T have to modify
their existing makefiles.

Because if they have to modify their makefiles anyway they might as
well just do the work to make them non-recursive in the first place.

> > However, I think this will be extremely difficult, because makefile
> > targets are so free-form and prerequisites and targets are not
> > actually necessary files at all.
> 
> Couldn’t these be handled by creating a "namespace" concept for
> rules? One could merge 2 Makefiles by prefixing each target with the
> directory of the Makefile. This would make dir1/all different from
> dir2/all. Also in recipes variables, pattern rules could be used from
> the Makefile describing the recipe. Would this not be possible (in
> theory, at least)?

Sure, but obviously it's not sufficient because some makefile rules DO
want to refer targets in other makefiles / directories.  I want my
target to depend on the "libfoo.a" target from some other directory. 
How does that work?

Also, how can you replace these target references if they appear inside
a recipe?  Make can't parse shell script content in recipes (if they
are even shell scripts at all; SHELL could be /usr/bin/perl or
something).  You can't just go through all the recipe text and replace
instances of the target "foo" with "dir1/foo".

We couldn't use a simple namespace naming convention like you suggest
of prefixing the directory because targets in the current makefile
could already be named "dir1/all" for example.  So you have to find a
way to talk about namespaces which don't conflict with existing naming.

Also what about things like make recipes that invoke recursive make,
_inside the same directory_.  Automake-generated makefiles do this all
the time.  If the namespace is based solely on the containing directory
is that sufficient?  Often the recursive invocation of make overrides
variables on the command line, so it seems like using the same
"namespace" wouldn't work.

And remember the premise here is that all these things have to work
WITHOUT any changes to makefiles (or Makefile.am files for automake)
themselves.

I'm not saying I know for a fact that it couldn't be done.  But it
definitely seems to me like there are lots of issues that have to be
considered and worked out, and I'm not really sure that it would really
be that much more efficient anyway.  The real efficiency gain would be
from:

(a) Not invoking multiple make processes at all... but the proposal on
the table is that we'd still invoke the make process, and it would
still parse its makefile, it would just ship the post-parsed content
back to the parent make, so that's at best equivalently efficient and
almost certainly slightly slower; and

(b) Allowing make to have a "global view" of all the rules and targets
so it can more efficiently run things in parallel... but here we're
talking about creating multiple namespaces anyway so can we really have
these multiple namespaces to separate target AND have this global view
of targets for better parallelism at the same time?

Probably what needs to happen to help answer these questions is for
someone (but, not me :)) to take a sample automake-enabled software
package, look at the makefiles generated by automake for that package,
and come up with a concrete proposal for how it might work and some
comments on how the result would be more efficient.