Re: [Help-bash] Closure concept in bash

[Bob Proulx]

Peng Yu wrote:
> Bob Proulx wrote:
> > Peng Yu wrote:
> >> See here for the explanation of closure. It is a concept in
> >> programming language. Most JavaScript code uses this property
> >> extensively. This feature (or at least something similar to it) will
> >> be also useful to bash in the case of "export -f func".
> >
> > Scoping in the shell is currently very simple.  At a first order of
> > approximation variables are either global or local and global variables
> > are either exported or not.
> >
> > The shell is very good at what it does.  If you want to do more than
> > what it does then you should use a different language such as awk,
> > perl, python, or ruby.  All of those have complex data structures
> > available to them.
> 
> Please take a look at the example code that uses gnu parallel in the
> first email.

I did look at it.  I just didn't comment upon it.  Exporting the world
into the environment space is a very hackish way to program in my not
so humble opinion.  Ugly as sin.  Blech!  Since the only comments I
could make about it were negative I didn't want to rain all over your
day with them.  And Greg made very good comments but didn't get
answers to his questions.  I would be asking the same questions.

Look at a rephrasing of one of your examples:

  #!/bin/bash
  function testing { echo test$1 ;}
  printf "x\ny\n" | parallel testing '{}'

You have defined a function "testing" in this script and then are
trying to call it from 'parallel' which will invoke a completely
different shell.  That feels awful to me.  Instead I would define them
in the script called from parallel.

But you have discovered that bash can "export" functions and store
them as text strings in exported variables.  Then later a different
bash subshell will parse the environment strings back into live
functions in a different shell.  Gack.  It works in a limited way but
the problems of trying to use that in a large scale way are terrible.

Some Problems:

Does 'parallel' call 'bash' or 'sh' later?  I don't know but probably
it calls sh since that is the standard thing to do.  Depending upon
the system sh won't understand those exported functions.  (Greg noted
this problem too.)

Text representations of programs take up a lot of memory space in the
environment.  And that memory space is parsed by bash later and stored
in internal form.  It is memory inefficient.

The name space is global.  If you have two programs both using this
technique then they can collide on names.  You would need to use a
naming convention (such as myproject_myfunctionname) in order to
organize around colliding with other uses of the same name.

Some Alternatives:

I am sure parallel is nifty and I don't want to disparage it but in
this context it isn't needed.  If you just want to launch programs in
the background you would be better off simply launching programs in
the background.  Within limits.  If I were running ten or even thirty
lightweight processes then I would simply launch them.

  printf "echo x\necho y\n" | while read cmd; do $cmd & done

I wouldn't do that if the input were going to be quite large.  Do not
run the large 100 case on a heavy process.  You will kill overwhelm
your machine.

Running ten lightweight jobs in the background is okay:

  seq --format="echo %g"  10 | while read cmd; do $cmd & done # okay

Running 100 or 10,000 jobs in the background all at once would be very
bad.  Do not do this.

  seq --format="echo %g" 100 | while read cmd; do $cmd & done # Bad!

Using parallel is probably better for the large number case of trying
to run a large number of tasks.  But using the shell with a while and
read means you have access to all of the functions in this file.

> Unfortunately, using other languages can not solve this
> problem. (Please correct me, it is the not the case.)

I have no idea what problem you are trying to solve.

Bob