Re: Quoting special characters (was: Re: Possible solution for special characters in makefile paths)

[Paul Smith]

On Sat, 2014-02-22 at 18:05 +0200, Eli Zaretskii wrote:
> > From: Paul Smith <address@hidden>
> > Cc: address@hidden
> > Date: Sat, 22 Feb 2014 10:23:33 -0500
> > 
> > So the first thing to decide is whether the quoting exists only internal
> > to make and is removed before the string is passed to any external
> > interface (shell, stat(), stdout, environment, etc.), or if the quoting
> > will be preserved and passed along to the commands make invokes (not
> > stat() obviously).
> > 
> > I think it's clear that the solution MUST only be internal to make.

> > Regardless of the quoting behaviors make employs, we cannot be sure that
> > whatever utility we use to run recipes will treat quotes the same way.
> > Users can choose whatever SHELL value they want, for example, and I
> > certainly would not want to provide a solution that worked only when a
> > POSIX-y shell was used.
> 
> That's not our problem.  This is a problem for the person who writes
> the Makefile which uses such strings.  We CANNOT solve this problem
> inside Make, so we shouldn't even try.  You already mentioned the
> problems with quoting in command lines -- these are unsolvable on the
> Make level.  Only the user knows how to solve them.

You can't just leave it up to the user!  Make has to INTERPRET these
things: internally it has to break up variables into distinct words.
That's the whole point.  If it was sufficient to leave it up to the user
then what we have today would work and there'd be no problems.

Suppose I have a variable like this:

  FOO = "foo bar" " biz " "foo\baz"

Now I write:

  all: $(FOO)
  $(FOO): ; @echo "'$(firstword $@)'"

What do we expect the output to be?

What if instead I do this:

  OUTPUT := $(patsubst %,echo '%' ;,$(FOO))
  all: ; @$(OUTPUT)

??

What if I do this:

  SHELL = /usr/bin/python
  all: $(FOO)
  $(FOO): ; @print """$@"""

??

> First, we don't _have_ to strip them; we could keep the quotes.  And
> second, we could strip them, but record in the variable that it was
> quoted.  In both cases, we will restore the quoting in the context
> that needs them, such as when they are part of a shell command line.

Restore what quoting?  What if SHELL is set to R, or Ruby, or Lisp, or
something with very different quoting rules than whatever make
understands?  That doesn't change the fact that make may have to chop
the strings up into words to do its job.

You're basically saying that make would need to understand the quoting
rules for any interpreter SHELL is set to, so that it can parse variable
values according to those rules when it needs to chop them into words,
and still provide the originally quoted string back to the interpreter
when it invokes SHELL.

I don't see how this is possible.  Make must have its own out-of-band
quoting syntax, because we cannot use the interpreter's quoting syntax.

> Why should we bother about these hypothetical use cases?  Are there
> any real-life Makefiles that do anything like that?  If there are,
> let's look at them, and let's understand why they need this quote
> juggling in the first place.

No matter how unlikely a given make sequence may look to us, someone,
somewhere will be doing it.  What we have to do is have a very precise
understanding of exactly what will break for a given change.  If it's
not possible to clearly articulate exactly what the backward
incompatibility is, then I definitely am leery of implementing it.

It's even better if make can detect the incompatibility and warn about
it or fail.

> > If make had a coherent backslash escaping mechanism (for example,
> > something like the shell where backslash always escapes the next
> > character and if you want a backslash you must always write "\\") then
> > we would be all set.  But instead make's implementation tries to be too
> > smart and only interprets backslashes as escapes when they appear in the
> > context of a special character...
> 
> Exactly my point: using a backslash to escape a blank is a natural
> extension of what we already do with a colon.  I fail to see why is it
> so different.

It's a huge compatibility problem.  Let's take a more realistic example.
Suppose you wanted to have a variable containing two words, "foo\" and
"bar".  Today you write this:

  FOO = foo\ bar

Now if you want to use this as a list of targets (we'll say they're
phony to avoid issues with real filenames, although "foo\" is a
perfectly valid POSIX filename):

  .PHONY: all $(FOO)
  all: ; $(FOO)
  $(FOO): ; @echo '$@'

Running "make 'foo\'" is perfectly valid.  And maybe you want to massage
these into URLs (having backslashes in URLs is pretty common, even
today):

  all: ; for a in $(patsubst %,'http:\\%',$(FOO)); do echo "$$a"; done


Now we come along with the next release of GNU make, and say that if a
space is prefixed with a backslash it's no longer a word separator, but
instead a literal space, and instead of two words, FOO is now the single
word "foo bar".  If you want "foo\" and "bar" you have to modify your
makefile:

  FOO = foo\\ bar

Right?