Build a JSON string with Bash variables

Question

I need to read these bash variables into my JSON string and I am not familiar with bash. any help is appreciated.

#!/bin/sh

BUCKET_NAME=testbucket
OBJECT_N         


        

Answer 1

You are better off using a program like jq to generate the JSON, if you don't know ahead of time if the contents of the variables are properly escaped for inclusion in JSON. Otherwise, you will just end up with invalid JSON for your trouble.

BUCKET_NAME=testbucket
OBJECT_NAME=testworkflow-2.0.1.jar
TARGET_LOCATION=/opt/test/testworkflow-2.0.1.jar

JSON_STRING=$( jq -n \
                  --arg bn "$BUCKET_NAME" \
                  --arg on "$OBJECT_NAME" \
                  --arg tl "$TARGET_LOCATION" \
                  '{bucketname: $bn, objectname: $on, targetlocation: $tl}' )

Answer 2

You can use printf:

JSON_FMT='{"bucketname":"%s","objectname":"%s","targetlocation":"%s"}\n'
printf "$JSON_FMT" "$BUCKET_NAME" "$OBJECT_NAME" "$TARGET_LOCATION"

much clear and simpler

Answer 3

A possibility:

JSON_STRING='{"bucketname":"'"$BUCKET_NAME"'","objectname":"'"$OBJECT_NAME"'","targetlocation":"'"$TARGET_LOCATION"'"}'

Answer 4

For Node.js Developer, or if you have node environment installed, you can try this:

JSON_STRING=$(node -e "console.log(JSON.stringify({bucketname: $BUCKET_NAME, objectname: $OBJECT_NAME, targetlocation: $TARGET_LOCATION}))")

Advantage of this method is you can easily convert very complicated JSON Object (like object contains array, or if you need int value instead of string) to JSON String without worrying about invalid json error.

Disadvantage is it's relying on Node.js environment.

Answer 5

If you need to build a JSON representation where members mapped to undefined or empty variables should be ommited, then jo can help.

#!/bin/bash

BUCKET_NAME=testbucket
OBJECT_NAME=""

JO_OPTS=()

if [[ ! "${BUCKET_NAME}x" = "x" ]] ; then
        JO_OPTS+=("bucketname=${BUCKET_NAME}")
fi

if [[ ! "${OBJECT_NAME}x" = "x" ]] ; then
        JO_OPTS+=("objectname=${OBJECT_NAME}")
fi

if [[ ! "${TARGET_LOCATION}x" = "x" ]] ; then
        JO_OPTS+=("targetlocation=${TARGET_LOCATION}")
fi

jo "${JO_OPTS[@]}"

Answer 6

I had to work out all possible ways to deal json strings in a command request, Please look at the following code to see why using single quotes can fail if used incorrectly.

# Create Release and Tag commit in Github repository

# returns string with in-place substituted variables 

json=$(cat <<-END
    {
        "tag_name": "${version}", 
        "target_commitish": "${branch}", 
        "name": "${title}", 
        "body": "${notes}", 
        "draft": ${is_draft}, 
        "prerelease": ${is_prerelease} 
    }
END
)

# returns raw string without any substitutions
# single or double quoted delimiter - check HEREDOC specs

json=$(cat <<-!"END"   # or 'END' 
    {
        "tag_name": "${version}", 
        "target_commitish": "${branch}", 
        "name": "${title}", 
        "body": "${notes}", 
        "draft": ${is_draft}, 
        "prerelease": ${is_prerelease} 
    }
END
)

# prints fully formatted string with substituted variables as follows:

echo "${json}"  
{ 
    "tag_name" : "My_tag", 
    "target_commitish":"My_branch"
    ....
}

Note 1: Use of single vs double quotes

# enclosing in single quotes means no variable substitution 
# (treats everything as raw char literals)

echo '${json}'   
${json} 

echo '"${json}"'   
"${json}" 

# enclosing in single quotes and outer double quotes causes
# variable expansion surrounded by single quotes(treated as raw char literals).

echo "'${json}'" 
'{ 
    "tag_name" : "My_tag", 
    "target_commitish":"My_branch"
    ....
}'

Note 2: Caution with Line terminators

• Note the json string is formatted with line terminators such as LF \n
• or carriage return \r(if its encoded on windows it contains CRLF \r\n)
• using (translate) tr utility from shell we can remove the line terminators if any

# following code serializes json and removes any line terminators 
# in substituted value/object variables too

json=$(echo "$json" | tr -d '\n' | tr -d '\r' )

# string enclosed in single quotes are still raw literals

echo '${json}'   
${json} 

echo '"${json}"'   
"${json}" 

# After CRLF/LF are removed

echo "'${json}'" 
'{ "tag_name" : "My_tag", "target_commitish":"My_branch" .... }'

Note 3: Formatting

• while manipulating json string with variables, we can use combination of ' and " such as following, if we want to protect some raw literals using outer double quotes to have in place substirution/string interpolation:

# mixing ' and " 

username=admin
password=pass

echo "$username:$password"
admin:pass

echo "$username"':'"$password"
admin:pass

echo "$username"'[${delimiter}]'"$password"
admin[${delimiter}]pass

Note 4: Using in a command

• Following curl request already removes existing \n (ie serializes json)

response=$(curl -i \
            --user ${username}:${api_token} \
            -X POST \
            -H 'Accept: application/vnd.github.v3+json' \
            -d "$json" \
            "https://api.github.com/repos/${username}/${repository}/releases" \
            --output /dev/null \
            --write-out "%{http_code}" \
            --silent
          )

So when using it for command variables, validate if it is properly formatted before using it :)