Re: [Help-bash] Does the parser backtrack?

[Dan Douglas]

On Tue, Oct 4, 2016 at 2:33 PM, Eric Blake <address@hidden> wrote:
> On 10/04/2016 09:22 AM, Daniel Martí wrote:
>> Telling the difference between $(( - arithmetic expansion - and $( ( -
>> subshell inside a command substitution - should be easy:
>
> No, it is absolutely hard, because it is context-dependent.
>
> POSIX itself says, at
> http://pubs.opengroup.org/onlinepubs/9699919799/xrat/V4_xcu_chap02.html
> C.2.6 Command Substitution:
>
> "Arithmetic expansions have precedence over command substitutions. That
> is, if the shell can parse an expansion beginning with "$((" as an
> arithmetic expansion then it will do so. It will only parse the
> expansion as a command substitution (that starts with a subshell) if it
> determines that it cannot parse the expansion as an arithmetic expansion."
> ...
> "The ambiguity is not restricted to the simple case of a single
> subshell. More complicated ambiguous cases are possible (even with just
> the standard shell syntax), such as:
>
> $(( cat <<EOH
> + ( (
> EOH
> ) && ( cat <<EOH
> ) ) + 1 +
> EOH
> ))
>
> This can be parsed as an arithmetic expansion, with cat and EOH as the
> names of shell variables.

It can't. It's usually impossible to determine the well-formedness of an
arithmetic expression at parse-time because the arithmetic grammar isn't
entirely part of the shell grammar. At least, not in any implementation
that I'm aware of.

Even though this example is slightly special in that it contains no word
expansions, the shell must still evaluate all word expansions prior to
tokenizing the arithmetic expression, by which point the overall type
of this expansion must already be determined. Since word expansions may
cause side-effects, the shell can't retroactively change its mind and
re-interpret the word as a command substitution by the time a concrete
syntax tree can be constructed.

The only reason to put a heredoc inside a command substitution is if
side-effects require the heredoc to be evaluated in a particular order or
as part of a certain subshell. Such constructions involving side-effects
from expansions or asynchronous operations require the expression to be
considered an arithmetic expansion.

Needless to say, a reasonable person would write something like this
instead:

$( (foo <<EOH1) && foo <<EOH2)
+ ( (
EOH1
) ) + 1 +
EOH2