Re: Possible solution for special characters in makefile paths

[Frank Heckenbach]

Some random comments:

> The advantages to this are (a) there is no change to the length of the
> string so the encoding can be performed in-place, and computing the size
> of an output buffer is trivial (it's the same size), and (b) there is no
> change needed to any existing tokenization in make, which is scanning
> for whitespace, parenthesis, braces, nul bytes, etc.: it will all
> continue to work with no changes.

An alternative, of course, would be a multi-character encoding, with
the advantage that all characters can be encoded uniquely (same
principle as UTF-8). Advantage (b) would still hold, but (a)
wouldn't, as strings would get longer. However, the growth would be
limited by a constant factor (2 ought to work), and decoding could
still be done in-place, as strings can only get shorter. Now, I
don't know how much additional effort this would be. From your
description it seems that decoding would occur in more places than
encoding, so it might be feasible ...

> We could make an attempt to fix them by modifying
> the built-in rules to use something like:
>
> %.o : %.c
>       $(COMPILE.o) -c '$<' -o '$@'
>
> Of course this fails if a target or prerequisite contains single quotes.

I should mention, as you're surely aware, that this problem already
exists WRT some other (non-whitespace) special characters, e.g.:

%:
        echo $@

Then "make 'a>b'" does "echo a>b" and overwrites b.

As someone who routinely quotes variables in shell scripts unless
there's a specific reason not to, I prefer to do the same in
Makefiles. As you point out, '${FOO}' only protects all special
characters except quotes, and "${FOO}" protects all except
double-quotes, backslashes and a few more, which is unintuitive if
you think of make recipes like shell script fragments too much. It
seems you can properly quote expressions for Bourne-compatible
shells using a substitution like this:

Q='$(subst ','\'',$(1))'

FOO=a"b'c\d$$e`f

foo:
        echo $(call Q,$(FOO))

However, that's very non-obvious for the average user (and as your
example shows, the problem will reach the average user when you
implement the escaping) and not very readable.

On the other hand, automatically doing this for built-in commands is
non-trivial too since other kinds of shells will probably require
different ways of quoting. (And it wouldn't help users write their
own recipes with proper quoting.)

> I have no 100% solution to these problems, other than the hope that
> paths containing single-quotes are far less abundant than paths
> containing whitespace (for example).

That's most likely the case. However, the problem with this kind of
argument is that as soon as the common case (whitespace) is handled,
some will, consciously or not, simply assume "special characters
work now" and get sloppy and not properly test such cases and sooner
or later produce some actually dangerous code ...

> 1. Any makefile that uses one of the chosen mapping characters will
>    fail.  We can detect this during makefile parsing and throw an
>    error, so this will not be a silent problem.

For my progress messages (cf.
http://lists.gnu.org/archive/html/bug-make/2013-04/msg00060.html), I
do indeed use \001 as an indicator (for basically the same reasons
you suggest to use it for encoding). It wouldn't be a major problem
for me to switch to some other indicator, so that's just to say that
based on my sample of size 1, such things do occur, portable or not.

In fact, what I do is to pass it in an environment variable, to be
able to run with or without progress messages as in this short
example (which only shows start messages; LOG_END is just the same
in this context). This indicates that environment variables
containing the mapping characters need to cause failure as well in
order to maintain strict backward-compatibility.

%:
        $(call LOG_START,Building $@)
        touch $@

# without messages
make foo

# with messages
LOG_START='@echo "\001 #progress-start# $(1)"' make foo

> 2. Any makefile using the chosen "quoting" token will break; i.e.
>    if some makefile today has "[ = foo" then uses "$[" later, and
>    we choose $[...] for quoting, this will fail.  It would have to
>    be changed to use "$([)" instead.  Same for ` if we choose
>    $`...` etc.

FWIW, I grepped my Makefiles for "$[" and the only occurrences I
found were bash arithmetic expansions like this:

SHELL=/bin/bash
foo:
        n=3; echo $$[$$n + 1]

I suppose that's harmless, since the "$" here is escaped and
wouldn't be parsed as part of "$[". Still, it would be worthwhile to
add a test of this kind if you implement this ...