Re: (maybe) zsh-related bug in tests/defs.in

[Stefano Lattarini]

At Tuesday 25 August 2009, Ralf Wildenhues <address@hidden> 
wrote:
> Hello Stefano,
>
> thanks for the report.
>
> > While trying to run the automake testsuite *in parallel*, using
> > zsh (version 4.3), I ran into some apparently random failures
> > (with most test scripts failing), which I tracked down to the
> > way $testSubDir is defined in `tests/defs.in'.
>
> Can you please specify how exactly you ran the tests using zsh,
$ cd tests && make -j 5 TEST_LOG_COMPILER=/bin/zsh check
> or got the build system to run the tests for you using zsh,
No, it didn't. It just (correctly) used /bin/sh.
> and which zsh version you are using?
 $ zsh --version
 zsh 4.3.10 (i686-pc-linux-gnu)
 $ dpkg -l zsh
 ...
 ii  zsh  4.3.10-3
 $ readlink /bin/sh
 bash
> Because if /bin/sh is zsh, or a symlink to zsh, then it should
> turn on 'emulate sh' mode all by itself already there (zsh
> checks its $0 on startup for matching 'sh' or > 'ksh').
>
> If you do something like
>   zsh $srcdir/all.test
> to run a test manually using zsh, then of course this won't work.
>
> But in that case I'd advise you to either not use zsh, or to add a
> symlink of the form
>   ln -s /path/to/zsh ./sh
>   ./sh $srcdir/all.test
You are perfectly right here: in this way, the error I found 
disappears.
By the way, to make it disappear, it's enough to activate the 
`NO_FUNCTION_ARGZERO' option of zsh *from the cmdline*, with e.g.:
  $ zsh --nofunctionargzero $srcdir/all.test
>
> If you instead do something like
>   make check SHELL=/path/to/zsh
>
> then that should have already failed much earlier: some of the
> constructs in the makefiles require sh emulation.
>
And this is true, also.

However, a very determined user can wreak havoc even when running
tests from just the makefile interface, e.g. by doing (as I wrote
before):
 $ make -j 5 TEST_LOG_COMPILER=/bin/zsh check
But he has to be very determined to broke things and to complicate
the life of automake's maintainers :-)

> > By default, in a sourced script, zsh set $0 to the path of that
> > same sourced script -- more info at:
> >   http://zsh.dotsrc.org/Doc/Release/Parameters.html#IDX166
> > This problem *seems* to be taken care of in `defs.in' itself, by
> > the command `emulate sh', but unfortunately this command is
> > issued too late.
> >
> > In fact, when `foo.test' includes `./defs', it isn't in "emulate
> > sh" mode yet, so that in `./defs' tha value of $0 remains set to
> > `./defs', no matter if the sh-compatibility mode is later
> > activated.  Thus $me is *always* set to `defs' and $testSubDir is
> > *always* set to `defs.dir'.  D'oh.
>
> Ouch.  Is there some zsh variable that retains the original name of
> the script?
Well, I don't know of any, and also googling I couldn't find anything
useful. Sorry.
>
> > I'm not sure if this can be truly classified as a bug, since the
> > test scripts all have a `#!/bin/sh' shebang line, and I hope
> > that few people or systems are crazy enough to use zsh as
> > their default Bourne shell.  Anyway, this problem might deserve
> > some consideration.
>
> Again I'm not yet convinced this happens in a real-world situation,
For what matters, after reading your considerations, I agree with you.
> but if it does, one thing we could do would be to check whether $0
> is ./defs or ./defs-p, and in that case add $$ or $RANDOM to allow
> for parallel builds to work.
Well, this would allow the test scripts to run in paraller, but if any
of them fails, its temporary files would be left in a directory whose
name has no clear relation with the test name.  Of course one can
still look for the "pwd" line in the corresponding *.log file to find
out the name of that directory, but this seems kind of awkward
(to me, at least).  However, awkward is better than broken.

- - -

Now I go for a final thought.

Since you provide no documented nor advised way to run your test
scripts with a shell different from /bin/sh, and since those scripts 
work well even if /bin/sh is a symlink to zsh, I am now convinced 
that there is *no* bug here. It was just myself pushing the 
testsuite's limits a bit too much.
If you want to err on the side of safety, you could add a warning in
tests/README stating not to use zsh to run the test scripts, unless
it's symlinked to /bin/sh or called with the `--nofunctionargzero' 
option.

Finally, just to be safe, I'll see what happens running:
 $ make -j5 TEST_LOG_COMPILER='zsh  --nofunctionargzero' check
on my system.

If any error comes up, I will start another thread on the list, 
reporting it.
>
> Cheers,
> Ralf
Thanks for your help and consideration,
   Stefano