Re: How does quote removal work with alternative forms of parameter expansion?

[Lawrence Velázquez]

On Sun, May 26, 2024, at 5:33 PM, Philippe Cerfon wrote:
> For the above, the current POSIX
> (https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_06_02)
> says:
> " Enclosing the full parameter expansion string in double-quotes shall
> not cause the following four varieties of pattern characters to be
> quoted, whereas quoting characters within the braces shall have this
> effect."
>
> Which I guess describes the above behavior, in other words: there is a
> 2nd level of quoting for the word part, right?

See Section 2.2.3, which covers the behavior of double-quoted
parameter expansions more verbosely.

        For the four varieties of parameter expansion that provide
        for substring processing (see Section 2.6.2), within the
        string of characters from an enclosed "${" to the matching
        '}', the double-quotes within which the expansion occurs
        shall have no effect on the handling of any special characters.

So in "${foo#bar}", special characters in `bar' retain their special
meanings.  This includes both pattern matching characters and quoting
characters.


> What I still don't get are the :- :+ etc.:
>
> For the the current POSIX says:
> "word shall be subjected to tilde expansion, parameter expansion,
> command substitution, and arithmetic expansion." so no quote removal,
> but the most recent draft already corrected that as an error and says:
> "word shall be subjected to tilde expansion, parameter expansion,
> command substitution, arithmetic expansion, and quote removal."

That only applies to unquoted parameter expansions.  Tilde expansion
and quote removal do not occur in double-quoted parameter expansions.

$ cat /tmp/foo.sh
unset foo
printf '%s %s\n' "${foo-~}" "${foo-'bar'}"
$ for sh in bash dash ksh mksh zsh; do "$sh" /tmp/foo.sh; done
~ 'bar'
~ 'bar'
~ 'bar'
~ 'bar'
~ 'bar'

Section 2.2.3 should clarify this.

        For parameter expansions other than the four varieties that
        provide for substring processing, within the string of
        characters from an enclosed "${" to the matching '}', the
        double-quotes within which the expansion occurs shall
        preserve the literal value of all characters, with the
        exception of the characters double-quote, backquote,
        <dollar-sign>, and <backslash>.

This requires that, for example, the single quotes in "${foo-'bar'}"
be treated literally rather than as quoting characters.

        If any unescaped double-quote characters occur within the
        string, other than in embedded command substitutions, the
        behavior is unspecified.

This leaves the behavior of "${foo-"bar"}" up to implementations.

        The backquote and <dollar-sign> characters shall follow the
        same rules as for characters in double-quotes described in
        this section.

This requires that command substitutions, parameter expansions, and
arithmetic expansions work as they usually do within double quotes.

        The <backslash> character shall follow the same rules as
        for characters in double-quotes described in this section
        except that it shall additionally retain its special meaning
        as an escape character when followed by '}' and this shall
        prevent the escaped '}' from being considered when determining
        the matching '}' (using the rule in Section 2.6.2).

This requires that backslashes work as they usually do within double
quotes, except that they also escape closing braces.  This means
that a backslash is not removed unless followed by a dollar sign,
backquote, backslash, newline, or closing brace.

$ unset foo
$ printf '%s\n' "${foo-bar\ baz}"
bar\ baz
$ printf '%s\n' "${foo-bar\$baz}"
bar$baz


> But above with ##, single quotes *did* prevent expansion. Why not here:
>
> x="${1:-'$bar'}"  |  bash x.sh ""   |   'expanded'  => why not quote
> removal, why expansion?

I don't know.  Bourne compatibility, perhaps.


> And what's even more weird, above, there is no quote removal (of the
> single quotes), but below there IS:
> $ printf '%s\n' "${unset:-$'$bar'}"
> expanded

I don't understand this specific behavior, but these quotes are not
removed in POSIX mode.

$ set -o posix
$ unset foo
$ bar=x
$ printf '%s\n' "${foo-$'$bar'}"
$'x'


> OTOH, zsh handles this a bit unexpected:
> % printf '%s\n' ${some_unset_var:-foo bar}
> foo bar
> (i.e. it makes one word out of foo bar and not two, as bash does:
> $ printf '%s\n' ${some_unset_var:-foo bar}
> foo
> bar

zsh does not word-split the results of unquoted parameter expansions
unless the ${=...} form is used or the SH_WORD_SPLIT option is set.

% printf '%s\n' ${foo-bar baz}
bar baz
% printf '%s\n' ${=foo-bar baz}
bar
baz

This doesn't have anything to do with what we're talking about.


> If I have:
> "${unset:-"$bar"}"
> How is that processed? Is it an inner word "$bar"
> and an outer  "${unset:-<result from word>}"
> or is it like three strings: "${unset:-"   $bar   "}"
> ?

If it were the latter, you'd expect the result of $bar to be subject
to filename expansion, but it isn't.


-- 
vq