Re: Output quoting (was: Re: Possible solution for special characters in makefile paths)

[Frank Heckenbach]

Sorry for the delay on my side too. As I said, I was quite busy, now
I'll probably have more time available again ...

Paul Smith wrote:

> I have two concerns about applying quoting automatically.
> 
> > %.x: %.y
> >         process $< -o $@
> > Which make internally would turn into:
> >         process $(call SHELL_QUOTE,$<) -o $(call SHELL_QUOTE,$@)
> 
> This is a simple case, but above you say "apply it automatically when
> substituting a variable in a recipe".  What if your variable contained
> an entire script; consider my example from my response to Eli:
> 
>    FOO = foo\ bar
>    BAR = case '$(FOO)' in (foo\ bar) echo $@;; esac
> 
>    $(FOO) : ; $(BAR)
> 
> When make goes to expand the variable reference $(BAR) does it run the
> equivalent of $(call SHELL_QUOTE,$(BAR))?  That can't work.

I hadn't considered this case (putting scripts in variables is
something even I do only rarely ;), and indeed, automatically
quoting would break this.

> Or does make automatically call SHELL_QUOTE only the automatic
> variables, and you have to do all the others by hand?  So then in the
> above example we'd have to write:
> 
>    BAR = case '$(call SHELL_QUOTE,$(FOO))' in (foo\ bar) echo $@;; esac
> 
> Is that the idea?

It wasn't my idea, but now it almost seems that's the best we can
do. I don't really like it, though, because it makes automatic
variables very different from other variables and it's quite a trap
for users to remember to add the SHELL_QUOTE call for the latter and
not the former (especially when changing recipes later).

(An alternative would be to always apply SHELL_QUOTE, but provide a
built-in function to explicitly suppress quoting (whose
implementation would be a nop, it would just stop make from applying
SHELL_QUOTE to its result). I see some pros and cons to it, but I
think the backward-incompatibily it would create already rules it
out, doesn't it?)

So at this point my conclusion is:

- encoded-strings without automatic shell-quoting is very
  cumbersome to use (putting an explicit quoting function around
  each variable use in each recipe if you want them to be
  special-character-safe),

- encoded-strings with automatic shell-quoting only for automatic
  variables is very tricky to use (see above),

- encoded-strings with automatic shell-quoting for all variables
  breaks things.

So I don't see a form of encoded-strings I really like and I do
prefer the backslash-keeping proposal now.

The following is just about implementation details for this proposal
that I don't support anymore, just for the record:

Eli Zaretskii wrote:

> > From: Frank Heckenbach <address@hidden>
> > 
> > If SHELL_QUOTE is applied automatically, the built-in rules need no
> > modification because they don't contain explicit quoting either.
> > 
> > Now, if you want to use another interpreter with different quoting
> > rules, you can do:
> > 
> > SHELL = /bin/frob
> > SHELL_QUOTE = $(subst or whatever ...)
> 
> OK, but that will make using SHELL_QUOTE mandatory (i.e. not
> optional), and we should require that users of non-default shells must
> set SHELL_QUOTE accordingly.

That was the idea.

> I'm OK with that, I just thought you had some suggestion that didn't
> require SHELL_QUOTE.  (I think that coming up with a correct
> SHELL_QUOTE definition in Make syntax for a given shell is a
> non-trivial job.)

Quite possibly. The definition I gave for POSIX sh was already a bit
involved. Make might provide built-in functions to quote for various
common shells (perhaps including POSIX sh, if only for efficiency).

Paul Smith wrote:

> The second issue I have is not so clear and may not be an issue but I
> worry about the effort needed to create constructed pathnames.  For
> example what if your rule says:
> 
>    SHELL = myinterpreter
>    SHELL_QUOTE = ...something...
> 
>    foo: ; foobar $(DIR)/$@
> 
> In this case if make automatically quoted each variable we'd get the
> equivalent of:
> 
>    foo: ; foobar $(call SHELL_QUOTE,$(DIR))/$(call SHELL_QUOTE,$@)
> 
> My concern is that for some shells doing this may be either very
> difficult or very inefficient, compared to running the quoting on a
> single "word" like this:
> 
>    foo: ; foobar $(call SHELL_QUOTE,$(DIR)/$@)
> 
> if you see what I mean.  I admit I don't have concrete example of how
> this could completely fail, but it could be complicated in something
> like Perl, for example, where you can't just concatenate strings by
> putting them next to each other; you have to use the "." operator.  So
> then would you have to do something like this:
> 
>    foo: ; foobar $(DIR).'/'.$@
> 
> instead of this:
> 
>    foo: ; foobar '$(DIR)/$@'

I give you the inefficiency (as said above, this could be mitigated
by having common forms of SHELL_QUOTE built-in).

As for the second issue, I think that's an interesting case that
shows that a SHELL_QUOTE function for Perl should not include the
outer pair of ''; it should just prefix each \ and ' by a \. So this
would actually be the recommended way for Perl:

    foo: ; foobar '$(DIR)/$@'
=>
    foo: ; foobar '$(call SHELL_QUOTE,$(DIR))/$(call SHELL_QUOTE,$@)'

So why it is different from sh? Because in sh, a ' within '' cannot
be quoted (unlike in Perl, C and many other languages).

In fact quoting is fundamentally different in sh vs. Perl, C etc.:
In the latter, quoting introduces string literals, whereas in sh,
everything is strings and quoting is just another way to escape
special characters. Therefore «echo foo» and «echo "foo"» are
equivalent in sh, but not in Perl. That might explain why having
quoting produce '' automatically is natural for sh, but requiring ''
explicitly is natural for Perl.

FWIW, in sh we could use "" instead (where " can be quoted) at the
expense of having more quoted characters, or no quotes and only
backslashes (at the expense of having many more quoted characters).

This seems a matter of taste. All I'm saying here is that
generalizing from sh to Perl (WRT '') is just (subtly) wrong here.