New escape method proposal (was: Re: Possible solution for special characters in makefile paths)

[Paul Smith]

On Sun, 2014-03-02 at 11:38 -0500, Paul Smith wrote:
> On Thu, 2014-02-20 at 03:22 -0500, Paul Smith wrote:
> > Hi all.
> 
> Thanks for participating so far.

Sorry for the delay.  Life is intruding.  I'd like to discuss the
various elements of the special character capability in different
threads.  As Eli has pointed out, these can be separated, for the most
part.  The part that I think has generated the most discussion so far is
the escaping input method, and I want to dive deep on the two proposals
and try to ensure all corner-cases are understood.

Let's leave aside the details of the encoding and the output quoting for
a future email.

This thread is for discussion of the original encoding proposal I made,
which was to introduce a new quoting capability, for example $[...] or
$'...'; however a function like $(quote ...) would also work of course.


The text (e.g., "..." above) would be encoded internally, such that any
character that was special to make (whitespace, ":", "$", "=", "#", ";"
for example) would not be treated specially in any make context at all
(targets, prerequisites, functions, etc.)  Then at the appropriate times
(just before the string needs to be provided back to the operating
system) it would be decoded.

In addition, the following would automatically be encoded by make:
      * The results of the $(wildcard ...) function
      * Any goal targets provided on the make command line

Environment variables which are special to make (MAKEFLAGS) could be
treated as pre-encoded, and when generated for sub-make invocation they
could be left encoded when placed into the environment, so that the
quoting can be preserved (this might need a bit of experimentation).


Encoded characters would not be considered to match the un-encoded
characters for the purposes of $(subst ...) etc.  So for example, given
this makefile:

   FOO = $[foo::bar]::baz

   X := $(subst   :,  -, $(FOO))
   Y := $(subst $[:], -, $(FOO))

$X will give "foo::bar--baz", and $Y will give "foo--bar::baz".


The only backward compatibility issue would be that "[" (or "'" or
whatever) would become an invalid variable name.  Any existing makefile
that assigned a variable with that name (e.g., "[ = ..." etc.) would
generate a failure during makefile parsing.

Even this backward-compat problem could be removed by using a new
function like $(quote ...) instead, at the cost of more verbose
makefiles.


If we were willing to take a bigger backward-compatibility hit, we could
completely remove the current special handling of backslash in makefiles
(see my email on this subject last week).  Instead of removing
backslashes, we'd treat them just like any other character.  This would
allow Windows pathname separators to be used without any special
treatment in makefiles.  We'd still need to treat the drive specifier
specially due to the colon.  Or we could require $[C:\Program Files\Foo]
in all cases, but that would be a BIG backward-compatibility break.

The current handling of backslashes is, IMO, really horrible and just
confusing and not very useful.  I'd love to get rid of it although this
would break all makefiles which currently stumble along using it, for
lack of anything better.  It might be something we'd have to deprecate
for a while before removal.


Advantages to this model:
     1. A brand-new syntax makes it very straightforward to understand
        what is quoted and what is not.
     2. Virtually no backward-compatibility issues.
     3. Large swaths of make internals can be used unmodified.
     4. Can co-exist, if we want, with current backslash handling in
        makefiles.

Disadvantages to this model:
     A. Uses a different quoting syntax than users are currently
        familiar with.
     B. Requires careful examination of code in make that interacts with
        the operating system to be sure we're decoding when needed.
     C. Will require some extra functions created for the C and Guile
        APIs so they can interact with encoded strings and decode them
        appropriately.