Re: Variable definition in eval'd function

[Paul Smith]

On Wed, 2007-10-10 at 16:35 -0400, Bryan Ischo wrote:
> LIBRARIES = libfoo libbar
> libfoo_SOURCES = foo.cpp foo2.cpp
> libbar_SOURCES = bar.cpp bar2.cpp

> In general, my approach is working well because all of my templates pass
> in the target names as template parameters, which *do* work the way I want
> in rules (I guess that variables of the form $(1) are "special" and when
> $(call) is invoked on a template, the variable expansion happens
> immediately even inside rules - I wish that were true for all variables!).

Sort of.  You have to keep the distinction between call and eval
straight.  Call will always expand all of its arguments; it is operating
on a string (the value of the first argument).  It doesn't parse the
string to decide that "oh this part of the string looks like it might be
a rule recipe so I'm not going to expand this bit", etc.  It's just a
variable value (string) and it all gets expanded.

That's why you need the extra $$ inside the recipes in the variable, to
keep call from expanding those.

Then the result of that expansion is passed to eval; eval DOES parse the
string as if it were a makefile, and so it DOES know that those lines
are actually recipe lines and it won't expand them.

> The only one that is giving me trouble is my "clean" rule, which is NOT
> generated by a template invocation, and thus does not use $(call)
> parameters in the rule.

There are a number of choices to solve this.  One simple one is to keep
your own internal variable for clean, that is appended to by the
template invocations, so something like in the template:

        _clean_LIBS += $$(LIBRARIES)

then in your clean rule you use $(_clean_LIBS) which will by then be a
complete list of the libraries to clean.

> However, I find that some aspects of how GNU make syntax works to be
> conter-intuitive and confusing.  Some of them probably are just me being
> dense (all of the double-escaping of variables necessary (i.e. $$(foo) so
> that $(eval $(call)) works as expected), and getting that wrong half the
> time),

I agree this is tricky to get right and I really regret this.
Unfortunately I just don't see a better way to do it.  I'm open to
suggestions, but note that I'd rather see the current behavior persist,
warts and all, then add some really different new capability that breaks
a lot of fundamental assumptions about how makefiles and expansion work.

> but the topic of this email in particular - the fact that variables
> are evaluated differently inside rules than outside rules - makes things
> much more difficult than necessary.
> 
> Does anyone know the reason behind this behavior?  I can't think of why
> doing it this way is better.  But I am sure I will be proven wrong as is
> often the case when I question the design of established software written
> by smart people :)

Well, I could be glib and point out that changing make to behave as you
suggest will not only violate the POSIX standard for make and cause GNU
make to behave differently than every other version of make in
existence, but it will immediately break at least 95% of the 10's or
probably 100's of thousands of makefiles that have been written since
make was first invented back in the 1980's! :-).

But, I will give you one real example of where NOT expanding variables
is crucial: consider automatic variables.  What if you write this:

        foo: bar
                @echo $^

        foo: biz baz
        foo: boz bez

?  With your change the output would be, best case, "bar", which is
clearly NOT what was intended.  It should be "bar biz baz boz bez".

Another implication of expansion being performed when the makefile is
read in is that every recipe is ALWAYS expanded; today that's not true.
In make today recipes are only expanded if they are actually going to be
run.  It is possible to imagine some (not very common, admittedly)
situations where this makes a difference: if the expansion has
side-effects.

Anyway, your assertion that expansion is handled differently inside and
outside recipes is not correct, or at least the truth is more subtle
than that.  Expansion in make is a function of where the variable
appears: in some places it is expanded immediately, and in some places
it is deferred.  Expansion of a deferred variable happens only when that
variable is used in an immediate context, some other place.

It's true that one place in makefile syntax where variable expansion is
deferred is within a recipe: in that case it's deferred until make wants
to run the recipe.  But, variable expansion is also deferred in
recursive variable assignments, which is why things like:

        COMPILE.c = $(CC) -o $@ -c $<

work; the right-hand side is NOT expanded here.

> Consider: if the variables in rules are only ever substituted once, when
> the rule is invoked (which I believe is what is happening right?) then any
> variable which is set more than once in a makefile will only ever use its
> final value in rules.  That means that something like this will never work
> the way I want:
> 
> CC = gcc
> foo: foo.c
>         $(CC) -o $@ $^
> CC = g++
> bar: bar.cpp
>         $(CC) -o $@ $^
> 
> This is a very contrived example - I know that in this particular example
> there would be many ways to work around this problem.  But in more complex
> examples, especially those using templates, it becomes much more difficult
> to work around.

As you say, this particular example is trivially handled via different
pattern rules for .c and .cpp files.  But even in the general case, this
is easily handled using either target-specific variables or, if you
prefer to be more portable, derived variables.

Target-specific variables solution:

        foo: CC = gcc
        foo: foo.c
                $(CC) -o $@ $^
        bar: CC = g++
        bar: bar.cpp
                $(CC) -o $@ $^

Derived variables solution:

        foo_CC = gcc
        foo: foo.c
                $(address@hidden) -o $@ $^
        bar_CC = g++
        bar: bar.cpp
                $(address@hidden) -o $@ $^

Either or both of these easily lend themselves to a templated
environment.  In fact I wrote a full environment very similar (even down
to syntax) to the one you're using and I found that derived variables
were VERY helpful in this model; I could do things like:

        LIBRARIES = foo

        foo_SRCS = foo.c bar.c baz.c
        foo_CFLAGS = -Wall -Werror
        bar.o_CFLAGS = -Wno-error

etc.

It's really important not to fall into the trap of reading makefiles as
if they were a procedural programming language like a shell script: they
are most definitely NOT.  Makefiles are essentially a declarative
programming language.

-- 
-------------------------------------------------------------------------------
 Paul D. Smith <address@hidden>          Find some GNU make tips at:
 http://www.gnu.org                      http://make.mad-scientist.us
 "Please remain calm...I may be mad, but I am a professional." --Mad Scientist