Re: GNU Parallel Bug Reports Unexpected execution of /bin/sh when SHELL or PARALLEL_SHELL is bash

[Zhiming Wang]

> On Nov 24, 2015, at 1:58 PM, Ole Tange <address@hidden> wrote:
> 
> On Fri, Nov 20, 2015 at 11:27 PM, Zhiming Wang <address@hidden> wrote:
> 
>> Consider the following script named 'test':
>> 
>>   #!/bin/bash
>>   func () {
>>       echo $BASH
>>       cat <(echo 'hello, world');
>>   }
>>   export -f func
>>   SHELL=/bin/bash parallel ::: func
>> 
>> When the script is executed in OS X (where /bin/sh is bash in sh emulation
>> mode; see
>> https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man1/sh.1.html):
>> 
>>> ./test
>>   sh: func: line 1: syntax error near unexpected token `('
>>   sh: func: line 1: ` cat <(echo 'hello, world')'
>>   sh: error importing function definition for `func'
>>   /usr/local/bin/bash
>>   hello, world
>> 
>> One can immediately spot two issues here:
>> 
>> 1.  Errors printed by sh. Bash-emulated sh tries to import exported functions
>>   anyway (which I see as a defect), but sh doesn't have process substitution,
>>   hence the errors. However, *why is /bin/sh called in the first place?*
> 
> /bin/sh is used for some internal calls: One of the calls is a call to
> `ps` to figure out the parent shell.
> 
> On my MacOSX I can get the same behaviour by simply:
> 
> #!/bin/bash
> 
> (echo "#!/bin/sh"; echo "echo foo") > /tmp/bar
> chmod 755 /tmp/bar
> # OK:
> /tmp/bar
> # Fails:
> func() {
> cat <(echo joe)
> }
> export -f func
> /tmp/bar

Correct, I detailed this behavior at
http://stackoverflow.com/a/33820568/1944784 (this problem was originally
reported on StackOverflow). Not so nice, but I guess we'll have to live with
it.

> When these internal calls are run the function is still exported. This
> confuses /bin/sh when hitting bash dialects. So GNU Parallel should
> probably provide /bin/sh with a clean environment for its internal
> calls - i.e. strip any functions for those calls.

`env -i` for internal calls, maybe? I don't know how much the internal calls
depend on existing environment, so this may or may not work. For ps it should
be fine.

>>   Note that 'func' is successfully executed despite the error messages, and
>>   the value of $BASH is /usr/local/bin/bash instead of /bin/sh, so bash is
>>   executed after all.
> 
> Yep. After running the internal calls to /bin/sh it starts bash.
> 
>>   Also note that the error messages are only reproducible on systems where
>>   /bin/sh is bash. For instance, Debian's /bin/sh is dash, which doesn't try
>>   to import exported functions in bash, so one won't see the error messages
>>   on Debian.
>> 
>> 2.  Although the parent shell and $SHELL are both /bin/bash, the shell that
>>   gets executed in the end is /usr/local/bin/bash,
> 
> Problem is here that `ps` does not give us full path to the shell: We
> only get the command name `-bash`. So GNU Parallel looks through your
> $PATH and the first command named `bash` it finds is apparently
> /usr/local/bin/bash.

It was somewhat surprising, but it makes sense now, thanks.

>> which seems to contradict
>>   the $PARALLEL_SHELL part of the man page, according to which the shell is
>>   determined in the following order:
>> 
>>       * $PARALLEL_SHELL;
>>       * The shell that started GNU Parallel (if it can be determined);
>>       * $SHELL;
>>       * /bin/sh.
> 
> Unless you have a way to get full path of the parent command then this
> part of the bug cannot be solved due to missing information from `ps`,
> so help me rephrase the man page, so the above behaviour is more
> clear.

I don't. Maybe just add a caveat in this part of the man page?

   The shell that started GNU Parallel (if it can be determined; note that GNU
   Parallel cannot possibly determine the full path of the parent shell, so at
   best the first shell on $PATH with the correct name is used);

Something like that?

Best,
Zhiming