Build a JSON string with Bash variables

Question

I need to read these bash variables into my JSON string and I am not familiar with bash. any help is appreciated.

#!/bin/sh

BUCKET_NAME=testbucket
OBJECT_N         


        

Answer 1

To build upon Hao's answer using NodeJS: you can split up the lines, and use the -p option which saves having to use console.log.

JSON_STRING=$(node -pe "
  JSON.stringify({
    bucketname: process.env.BUCKET_NAME,
    objectname: process.env.OBJECT_NAME,
    targetlocation: process.env.TARGET_LOCATION
  });
")

An inconvenience is that you need to export the variables beforehand, i.e.

export BUCKET_NAME=testbucket
# etc.

---

Note: You might be thinking, why use process.env? Why not just use single quotes and have bucketname: '$BUCKET_NAME', etc so bash inserts the variables? The reason is that using process.env is safer - if you don't have control over the contents of $TARGET_LOCATION it could inject JavaScript into your node command and do malicious things (by closing the single quote, e.g. the $TARGET_LOCATION string contents could be '}); /* Here I can run commands to delete files! */; console.log({'a': 'b. On the other hand, process.env takes care of sanitising the input.

Answer 2

First, don't use ALL_CAPS_VARNAMES: it's too easy to accidentally overwrite a crucial shell variable (like PATH)

Mixing single and double quotes in shell strings can be a hassle. In this case, I'd use printf:

bucket_name=testbucket
object_name=testworkflow-2.0.1.jar
target_location=/opt/test/testworkflow-2.0.1.jar
template='{"bucketname":"%s","objectname":"%s","targetlocation":"%s"}'

json_string=$(printf "$template" "$BUCKET_NAME" "$OBJECT_NAME" "$TARGET_LOCATION")

echo "$json_string"

For homework, read this page carefully: Security implications of forgetting to quote a variable in bash/POSIX shells

---

A note on creating JSON with string concatenation: there are edge cases. For example, if any of your strings contain double quotes, you can broken JSON:

$ bucket_name='a "string with quotes"'
$ printf '{"bucket":"%s"}\n' "$bucket_name"
{"bucket":"a "string with quotes""}

Do do this more safely with bash, we need to escape that string's double quotes:

$ printf '{"bucket":"%s"}\n' "${bucket_name//\"/\\\"}"
{"bucket":"a \"string with quotes\""}

Answer 3

You could use envsubst:

  export VAR="some_value_here"
  echo '{"test":"$VAR"}' | envsubst > json.json

also it might be a "template" file:

//json.template
{"var": "$VALUE", "another_var":"$ANOTHER_VALUE"}

So after you could do:

export VALUE="some_value_here"
export ANOTHER_VALUE="something_else"
cat  json.template | envsubst > misha.json

Answer 4

can be done following way:

JSON_STRING='{"bucketname":"'$BUCKET_NAME'","objectname":"'$OBJECT_NAME'","targetlocation":"'$TARGET_LOCATION'"}'

Answer 5

Bash will not insert variables into a single-quote string. In order to get the variables bash needs a double-quote string. You need to use double-quote string for the JSON and just escape double-quote characters inside JSON string. Example:

#!/bin/sh

BUCKET_NAME=testbucket
OBJECT_NAME=testworkflow-2.0.1.jar
TARGET_LOCATION=/opt/test/testworkflow-2.0.1.jar

JSON_STRING="{\"bucketname\":\"$BUCKET_NAME\",\"objectname\":\"$OBJECT_NAME\",\"targetlocation\":\"$TARGET_LOCATION\"}"


echo $JSON_STRING 

Answer 6

These solutions come a little late but I think they are inherently simpler that previous suggestions (avoiding the complications of quoting and escaping).

    BUCKET_NAME=testbucket
    OBJECT_NAME=testworkflow-2.0.1.jar
    TARGET_LOCATION=/opt/test/testworkflow-2.0.1.jar
    
    # Initial unsuccessful solution
    JSON_STRING='{"bucketname":"$BUCKET_NAME","objectname":"$OBJECT_NAME","targetlocation":"$TARGET_LOCATION"}'
    echo $JSON_STRING 
    
    # If your substitution variables have NO whitespace this is sufficient
    JSON_STRING=$(tr -d [:space:] <<JSON
    {"bucketname":"$BUCKET_NAME","objectname":"$OBJECT_NAME","targetlocation":"$TARGET_LOCATION"}
    JSON
    )
    echo $JSON_STRING 
    
    # If your substitution variables are more general and maybe have whitespace this works
    JSON_STRING=$(jq -c . <<JSON
    {"bucketname":"$BUCKET_NAME","objectname":"$OBJECT_NAME","targetlocation":"$TARGET_LOCATION"}
    JSON
    )
    echo $JSON_STRING 
    
    #... A change in layout could also make it more maintainable
    JSON_STRING=$(jq -c . <<JSON
    {
       "bucketname" : "$BUCKET_NAME",
       "objectname" : "$OBJECT_NAME",
       "targetlocation" : "$TARGET_LOCATION"
    }
    JSON
    )
    echo $JSON_STRING