help-bash
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Help-bash] Readig multiple null-separated values


From: František Kučera
Subject: Re: [Help-bash] Readig multiple null-separated values
Date: Thu, 1 Aug 2019 15:55:03 +0200
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.8.0

Dne 29. 07. 19 v 15:07 Greg Wooledge napsal(a):
In Bash 4.4, the builtin `mapfile' has an opton `-d delim' which
enables to change the delimiter from newlines. It accepts an empty
string which actually behaves as NUL separator. For example, the
provided example can be written with `mapfile' as follows:

   $ printf 'a\0aaa\0b\0bbb\0c\0ccc' | { mapfile -d '' arr; declare -p arr; }
   declare -a arr=([0]="a" [1]="aaa" [2]="b" [3]="bbb" [4]="c" [5]="ccc")
And prior to bash 4.4, the only choice was IFS= read -r -d '' which works
basically the same way, just with string variables instead of array
variables.

$ for var in a b c; do IFS= read -r -d '' "$var"; done < <(printf 
'a\0aaa\0b\0bbb\0c\0ccc\0')

Thanks for responses. This is similar to my function, just in-lined.

The mapfile can read null-separated values, but it reads them into a single-dimensional array – but the logical structure in my case is not an array but a two-dimensional table (rows, colums) and it is also useful to reference the values by their names instead of numeric indexes.

My first attempt was to convince the read command to read e.g. three null-separated into three variables (the first record/row) and then stop, so the "while" body will be executed, and in the next cycle the read would read the next three values (second record) into the variables. But this does not work, because if the delimiter is set to \0, the read stops reading even if there are more variables waiting to be filled. It is good default behavior but there might be an option to read all variables even if the delimiter occurred (i.e. option to continue over delimiters until all variables are filled and then return).

Since the OP seems to be asking for a Swiss Army Knife that handles either
arrays or strings (and opens wine bottles too), the only answer we can
give is: there is no single tool.  What you use depends on the situation.

  * If you need a fixed number of string values, you can simply call
    read however many times you require.

  * If you need an array, and you're in bash 4.4 or newer, you can use
    mapfile.

  * If you need an array, and you're targeting older bash releases, you
    can call read inside a while loop, and build up the array that way.

To illustrate a use-case – I sometimes pass a stream of null-separated values to this function from Relational pipes <https://relational-pipes.globalcode.info/> if I need to process such data in a script, e.g. call some shell commands for each record. For example:

find -type f -print0 | relpipe-in-filesystem --file path --xattr xdg.origin.url | relpipe-out-nullbyte | while read_nullbyte file url; do echo "File $file was downloaded from $url and has SHA-256 hash: $(sha256sum "$file")"; done

Echoing the string is too simple task – just for illustration – inside the for cycle there might be any other Bash code which works with variables of current record loaded from the stream. The data can come from various sources like XML, Recfiles, databases etc. and can be filtered or JOINed in particular pipeline steps. It is similar to declarative SQL, but without database/server and it is stream-oriented.

Franta




reply via email to

[Prev in Thread] Current Thread [Next in Thread]