[Help-bash] "too much" loop overhead

[Peng Yu]

Hi,

The following are two pairs of equivalent bash and python scripts.

main.* shows that bash is slower than python. However, the main_loop.*
indicates that the slow down is most likely from the foo-loop.

./main_colon.sh means that  ":" x 10000 in bash takes about .03 sec
runtime. So the bash for-loop takes about 0.127 - .03 =~ 0.09, whereas
the python for-loop takes .001 sec.

Removing the time of the loop, the bash realpath builtin is actually
faster. After all, the realpath builtin is just a dynamically linked
library, it is unceiveable that python would do it even better than
native C.

The above calculation is based on the assumption that different
statements in a script do not interfere with each other in terms of
the runtime. Let me know if this assumption is not true. It is not
clear how to just measure the runtime of the builtin with the loop
time accurately removed.

Anyway, the test shows that the loop overhead is even comparable to a
builtin function that does the actual work. This sounds to be an
overhead that is not very acceptable.

What is the reason that the loop in bash is so slow? How to make it
comparable at least to python if not to the native C code? Thanks.

$ ./main.sh > /dev/null
0.331
$ ./main.py > /dev/null
0.300492048264
$ cat main.sh
#!/usr/bin/env bash
# vim: set noexpandtab tabstop=2:

enable -f ./realpath realpath
TIMEFORMAT=%R
time for ((i=0;i<10000;++i)); do
    realpath .
done
$ cat main.py
#!/usr/bin/env python
# vim: set noexpandtab tabstop=2 shiftwidth=2 softtabstop=-1 fileencoding=utf-8:

from os.path import abspath
import timeit
import sys
def f():
    for i in range(10000):
        print abspath('.')
print >> sys.stderr, timeit.timeit('f()', number=1, setup="from
__main__ import f")

$ cat main_loop.py
#!/usr/bin/env python
# vim: set noexpandtab tabstop=2 shiftwidth=2 softtabstop=-1 fileencoding=utf-8:

import timeit
import sys
def f():
    for i in range(10000):
        pass
print >> sys.stderr, timeit.timeit('f()', number=1, setup="from
__main__ import f")

$ ./main_loop.py
0.00102996826172
$ cat ./main_colon.sh
#!/usr/bin/env bash
# vim: set noexpandtab tabstop=2:

TIMEFORMAT=%R
printf '%s\t' 1; { time for ((i=0;i<10000;++i)) do :; done; } 2>&1
printf '%s\t' 2; { time for ((i=0;i<10000;++i)) do :;:; done; } 2>&1
printf '%s\t' 3; { time for ((i=0;i<10000;++i)) do :;:;:; done; } 2>&1
printf '%s\t' 4; { time for ((i=0;i<10000;++i)) do :;:;:;:; done; } 2>&1
printf '%s\t' 5; { time for ((i=0;i<10000;++i)) do :;:;:;:;:; done; } 2>&1
printf '%s\t' 6; { time for ((i=0;i<10000;++i)) do :;:;:;:;:;:; done; } 2>&1
printf '%s\t' 7; { time for ((i=0;i<10000;++i)) do :;:;:;:;:;:;:; done; } 2>&1
$  ./main_colon.sh
1    0.127
2    0.163
3    0.196
4    0.229
5    0.267
6    0.309
7    0.333

-- 
Regards,
Peng