Re: [Help-bash] Question about escaping in backquote string in double quoted strings

[Chet Ramey]

On 5/2/19 1:05 PM, KHMan wrote:
> Hi all,
> 
> I have a question about escaping of the double quote char in a backquote
> command string nested inside a double quoted string. I have been trying out
> an example from the Morbig paper[1] and looking at the bash ref man but
> couldn't figure things out.
> 
> Here is a slightly adjusted example from page 4, Example 2.5. Output is at
> right after the comment #.
> 
> echo "[` echo \" \\" \" `]"             # [ " ]      (A)

OK, let's look at A. I have simplified the explanation by breaking things
into separate steps, even if the code itself performs multiple steps in a
single pass. All the quoted text snippets are from the POSIX standard.

The parser takes this and applies the double-quoting rules. The command
substitution rules enter the picture because POSIX leaves enough behavior
undefined so that the double-quoted string can be terminated by a double
quote in the backquoted command substitution (ksh93 does this, for
instance).

The bash parser doesn't allow quotes in a command substitution to terminate
a double-quoted string that was started outside it, and doesn't remove
backslashes until word expansion, so we have

"[` echo \" \\" \" `]"

as the word to be expanded.

That double-quoted string is processed according to the POSIX rules, and we
end up with

[` echo " \" " `]

because this double-quoted string rule is in effect:

"The <backslash> shall retain its special meaning as an escape character
(see Escape Character (Backslash)) only when followed by one of the
following characters when considered special:

$   `   "   \   <newline>"


Those rules mean that the double quote embedded within the backquoted
command substitution has the backslash removed; similarly for the backslash
quoting the backslash.

At this point, you're using the rules for double-quoted strings to find
the ending backquote, but there's no complication with that in this
example. This string gets expanded, left-to-right.

The non-special `[' and `]' characters can be ignored; you just add them
to the result string when you encounter them.

Now you can process the command substitution. You have to make a pass to
remove the escapes that the backquoted form of command substitution
considers special, because

"The portion of the quoted string from the initial backquote and the
characters up to the next backquote that is not preceded by a <backslash>,
having escape characters removed, defines that command whose output
replaces "`...`" when the word is expanded."

Since there aren't any backslashes quoting "$", "`", or "\", this step
has no effect, and the command substitution consists of

echo " \" "

which goes through normal parsing and execution, and results in ` " '. That
string gets run through the remaining parts of word expansion relevant to
double-quoted words (basically none), and you end up seeing

[ " ]


> echo ` echo \" \\" \" `                 # " " "      (B)

In this case, you don't have the double-quoted string rules to worry about;
the command to be substituted ends up being

echo \" \" \"

because of the step that removes escape characters special to the
backquotes noted above.

> 
> In section 3.5.4 Command Substitution, no mention about escaping double
> quotes is made. I understand (B) but not (A). Who processed the \" into "
> in (A), because when I tried the $(command) form, no escape processing was
> done whether the command string is on its own or nested inside a double
> quoted string (C)-(F):

The $(...) form has different rules, and there's no special processing
performed on the text between the parens:

"With the $(command) form, all characters following the open parenthesis to
the matching closing parenthesis constitute the command. Any valid shell
script can be used for command, except a script consisting solely of
redirections which produces unspecified results."

and

"The input characters within the quoted string that are also enclosed
between "$(" and the matching ')' shall not be affected by the double-
quotes, but rather shall define that command whose output replaces the
"$(...)" when the word is expanded."

Chet

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
                 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU    address@hidden    http://tiswww.cwru.edu/~chet/