Re: [Help-bash] split and substitute

[Greg Wooledge]

On Sat, Jun 11, 2016 at 01:55:59AM +0000, Val Krem wrote:
> Hi all,
> 
> I have two data sets (my data set and  reference file)
[...]

At this point, you really need to learn how to approach this kind of
problem.  You've introduced several variants of this already, so it
would be in your best interest to learn the basic programming techniques
that will help you solve them yourself.

Today's lesson is the associative array.

An associative array is an array that is indexed by arbitrary strings,
instead of by integers.  In bash 4, they are created with the command

declare -A arrayname

Then, indexing is just like with ordinary arrays, but using strings.
For example, to read all of the "references" from file2:

declare -A ref
while read -r key value; do
    ref["$key"]=$value
done < "file2"

Once those are read into memory, you can iterate over file1 in a
similar way.  For each key you read from file1, look for it in the
associative array and act on whatever you find (or do not find).

For example, to test for existence, we simply assume that none of the
values can be the empty string, and then:

if [[ ${ref["$key"]} ]]; then
    echo "key exists in ref"
else
    echo "key does not exist in ref"
fi

(Some versions of bash can also use test -v on an array element to test
for existence, but a simple string length check is more likely to work.)

For simple problems of this form, you could also use awk.  Awk's arrays
are associative arrays, and can be used in the same way (but without
the declaration).  Most other high-level scripting languages have them
too.  Sometimes they're called "hashes" or "dictionaries".  If you find
that your bash scripts are too slow, moving to a different language is
usually your best choice.