Parse out key=value pairs into variables

Question

I have a bunch of different kinds of files I need to look at periodically, and what they have in common is that the lines have a bunch of key=value type strings

Answer 1

I know this is particularly regarding awk but mentioning this as many people come here for solutions to break down name = value pairs ( with / without using awk as such).

I found below way simple straight forward and very effective in managing multiple spaces / commas as well -

Source: http://jayconrod.com/posts/35/parsing-keyvalue-pairs-in-bash

change="foo=red bar=green baz=blue"

#use below if var is in CSV (instead of space as delim)
change=`echo $change | tr ',' ' '`

for change in $changes; do
    set -- `echo $change | tr '=' ' '`
    echo "variable name == $1  and variable value == $2"
    #can assign value to a variable like below
    eval my_var_$1=$2;
done

Answer 2

$ cat file | sed -r 's/[[:alnum:]]+=/\n&/g' | awk -F= '$1=="Var"{print $2}'
Howdy Other

Or, avoiding the useless use of cat:

$ sed -r 's/[[:alnum:]]+=/\n&/g' file | awk -F= '$1=="Var"{print $2}'
Howdy Other

How it works

• sed -r 's/[[:alnum:]]+=/\n&/g'

  This places each key,value pair on its own line.

• awk -F= '$1=="Var"{print $2}'

  This reads the key-value pairs. Since the field separator is chosen to be =, the key ends up as field 1 and the value as field 2. Thus, we just look for lines whose first field is Var and print the corresponding value.

Answer 3

The closest you can get is to parse the variables into an associative array first thing every line. That is to say,

awk '{ delete vars; for(i = 1; i <= NF; ++i) { n = index($i, "="); if(n) { vars[substr($i, 1, n - 1)] = substr($i, n + 1) } } Var = vars["Var"] } { print Var, $5 }'

More readably:

{
  delete vars;                   # clean up previous variable values
  for(i = 1; i <= NF; ++i) {     # walk through fields
    n = index($i, "=");          # search for =
    if(n) {                      # if there is one:

                                 # remember value by name. The reason I use
                                 # substr over split is the possibility of
                                 # something like Var=foo=bar=baz (that will
                                 # be parsed into a variable Var with the
                                 # value "foo=bar=baz" this way).
      vars[substr($i, 1, n - 1)] = substr($i, n + 1)
    }
  }

  # if you know precisely what variable names you expect to get, you can
  # assign to them here:
  Var     = vars["Var"]
  Version = vars["Version"]
  Len     = vars["Len"]
}
{
  print Var, $5                  # then use them in the rest of the code
}

Answer 4

I will try to explain you a very generic way to do this which you can adapt easily if you want to print out other stuff.

Assume you have a string which has a format like this:

key1=value1 key2=value2 key3=value3

or more generic

key1_fs2_value1_fs1_key2_fs2_value2_fs1_key3_fs2_value3

With fs1 and fs2 two different field separators.

You would like to make a selection or some operations with these values. To do this, the easiest is to store these in an associative array:

array["key1"] => value1
array["key2"] => value2
array["key3"] => value3
array["key1","full"] => "key1=value1"
array["key2","full"] => "key2=value2"
array["key3","full"] => "key3=value3"

This can be done with the following function in awk:

function str2map(str,fs1,fs2,map,   n,tmp) {
   n=split(str,map,fs1)
   for (;n>0;n--) { 
     split(map[n],tmp,fs2);
     map[tmp[1]]=tmp[2]; map[tmp[1],"full"]=map[n]
     delete map[n]
   }
}

So, after processing the string, you have the full flexibility to do operations in any way you like:

awk '
    function str2map(str,fs1,fs2,map,   n,tmp) {
       n=split(str,map,fs1)
       for (;n>0;n--) { 
         split(map[n],tmp,fs2);
         map[tmp[1]]=tmp[2]; map[tmp[1],"full"]=map[n]
         delete map[n]
       }
    }
    { str2map($0," ","=",map) }
    { print map["Var","full"] }
   ' file

The advantage of this method is that you can easily adapt your code to print any other key you are interested in, or even make selections based on this, example:

(map["Version"] < 3) { print map["var"]/map["Len"] }

Answer 5

Since discussion in commentary has made it clear that a pure-bash solution would also be acceptable:

#!/bin/bash
#      ^-- must be /bin/bash, not /bin/sh
#
# ...must also be bash 4.0 or newer.

while read -r -a words; do                # iterate over lines of input
  declare -A vars=( )                  # refresh variables for each line
  set -- "${words[@]}"                 # update positional parameters
  for word; do
    if [[ $word = *"="* ]]; then       # if a word contains an "="...
       vars[${word%%=*}]=${word#*=}    # ...then set it as an associative-array key
    fi
  done
  echo "${vars[Var]} $5"              # Here, we use content read from that line.
done <<<"Version=2 Len=17 Hello Var=Howdy Other"

The <<<"Input Here" could also be <file.txt, in which case lines in the file would be iterated over.

If you wanted to use $Var instead of ${vars[Var]}, then substitute printf -v "${word%%=*}" %s "${word*=}" in place of vars[${word%%=*}]=${word#*=}, and remove references to vars elsewhere. Note that this doesn't allow for a good way to clean up variables between lines of input, as the associative-array approach does.