Re: [Help-bash] databases for bash and other data structures

[John Kearney]

Actually I had a lot of success with awk. :)

its pretty quick. 

you basically have a 2 dimensional array in a sort of csv format. but I chose eliminators so I could multi line entries for each element. No reason why you couldn't have up to 4 dimensions pretty easily. but it gets more tricky and normally you don't really need it. 

Firstly I found using compression improves performance.
so
  ks_awkdb_Awk() {
    local FName="${1%.gz}"
    shift
    if [ -z "${FName:-}" ]; then
      awk  "address@hidden" 

    elif [ -f "${FName}.gz" ]; then
      ks_log_Info "${LINENO:-}" 4 gzip -dc "${FName}.gz" \| awk  "address@hidden"
      gzip -dc "${FName}.gz" | awk  "address@hidden"
    elif [ -f "${FName}" ]; then
      ks_log_Info "${LINENO:-}" 5  awk  "address@hidden" "${FName}"
      awk  "address@hidden" "${FName}"
    else
      ks_log_Error "${LINENO:-}" Can\'t Find File Unknown Command "(${FName})"

    fi
  }
  ks_awkdb_RowSeperator="${RS}"
  ks_awkdb_UnitSeperator="${US}"

  ks_awkdb_Transcode='
  BEGIN {
    RS = "'"${ks_awkdb_RowSeperator}"'" 

    FS = "'"${ks_awkdb_UnitSeperator}"'" 
    ORS = "'"${ks_awkdb_RowSeperator}"'" 
    OFS = "'"${ks_awkdb_UnitSeperator}"'" 
  }
  '
to return a row or rows you can use something like
  ks_awkdb_GetDbRow_Match_c() {
    ks_awkdb_Awk "${2?Missing Db File}" "${ks_awkdb_Transcode}"' 
  '"${1:?Missing Match Condition}"'{ 

          gsub(FS, OFS, $0 ); 
          print NR, $0
          ECode=0;
          exit 0;
      }
      BEGIN  { ECode=2 } END { exit ECode }'
  }
  ks_awkdb_GetDbRow_Match() {
    local CLine 

    local Idx=$((${3:-1}-1))
    CLine="$(ks_awkdb_GetDbRow_Match_c "${2:?Missing Match Condition}" "${4?Missing Db File}" )" || return $?
    ks_awkdb_SplitRows2Array "CLine" "${CLine}" 

    if [ ${Idx} -ge 0 ]; then
      [ ${Idx} -lt address@hidden ] || return 1
      ks_awkdb_SplitRow2Array "${1}" "${CLine[${Idx}]:-}" 
    elif  [ $((address@hidden)) -ge 0 ]; then
      ks_awkdb_SplitRow2Array "${1}" "address@hidden:-}" 

    else
      return 1
    fi
    return 0
  }

The Match condiditon can be anything that awk supports.
so 
NR==5, NR==10   rows 5 to 10.
$4=="MatchString"  where column 4 is that string value.

$4~=//                    regex against column 4
 
I actually ended up with a pretty reliable and surprisingly quick system. I mainly use it read only. but when I'm creating it I do modify entries. because of how sed and awk work its actually pretty easy to do that in place. 

cheers

On Wed, Mar 13, 2013 at 7:35 AM, Chris Down <address@hidden> wrote:

On 2013-03-12 23:30, Jesse Molina wrote:
> Chris Down wrote:
> >Have you considered using flat files with a directory hierarchy?
>
> I had already written a little csv table with multiple rows.
> Iterating through the file with "read", until I found the right row
> via index and then "read COLUMN1 COLUMN2 COLUMN3..." is easy enough,
> if performance abhorrent, but updating the rows just got ugly and
> from there I decided that stabbing myself in the eye with a fork
> wasn't a good idea.

Well, you're using the wrong tool for the job; it's to be expected. I am only
holding myself back from having a nervous breakdown about the idea because you
stated that this is simply an educational exercise :-)

Chris

ks_mod_AwkDb.sh
Description: Bourne shell script