Parse or view JSON data fields using JQ tool utility where field names have a “-” dash in the key name

Question

I have a JSON data file (as shown below) and I'm trying to find field values using jq utility.

It's working fine except for fields if the key name contains a - dash character in it.

How can I get the values of "field-2", "field-three" or "field-three.url" for element under content.book1 (using jq at least)?

I tried the following to get the values but it's giving me the following errors for fields whose key name contains a dash - in it's name. I tried to back slash - character but that didn't help either.

Error types found:

jq: error (at <stdin>:27): null (null) and number (2) cannot be subtracted
jq: 1 compile error

jq: error: three/0 is not defined at <top-level>

jq: error: syntax error, unexpected INVALID_CHARACTER, expecting $end (Unix shell quoting issues?) at <top-level>

jq: error: syntax error, unexpected INVALID_CHARACTER, expecting FORMAT or QQSTRING_START (Unix shell quoting issues?) at <top-level>, line 1:

Commands:

$ cat /tmp/my.data.json
{
  "pages": {
    "book1": [
      "page1",
      "page2-para1",
      "page3-para1-sentence1",
      "page3-para2-sentence3-word4"
    ]
  },
  "content": {
    "book1": {
      "name": "giga",
      "url": "-",
      "field1": "value1",
      "field-2": "value-2",
      "field-three": {
        "name": "THIRD",
        "url": "book1/field-three/",
        "short-url": "book1/field-three/chota-chetan"
      },
      "authur": {
        "name": "lori CHUCK",
        "displayIndex": 4
      },
      "route": "/in-gc/hindi-chini-bhai-bhai"
    }
  }
}

$ cat /tmp/my.data.json| jq ".pages"
{
  "book1": [
    "page1",
    "page2-para1",
    "page3-para1-sentence1",
    "page3-para2-sentence3-word4"
  ]
}

$ cat /tmp/my.data.json| jq ".pages.book1[0]"
"page1"

$ cat /tmp/my.data.json| jq ".pages.book1[1]"
"page2-para1"

$ cat /tmp/my.data.json| jq ".content"
{
  "book1": {
    "name": "giga",
    "url": "-",
    "field1": "value1",
    "field-2": "value-2",
    "field-three": {
      "name": "THIRD",
      "url": "book1/field-three/"
    },
    "authur": {
      "name": "lori CHUCK",
      "displayIndex": 4
    },
    "route": "/in-gc/hindi-chini-bhai-bhai"
  }
}

$ cat /tmp/my.data.json| jq ".content.book1"
{
  "name": "giga",
  "url": "-",
  "field1": "value1",
  "field-2": "value-2",
  "field-three": {
    "name": "THIRD",
    "url": "book1/field-three/"
  },
  "authur": {
    "name": "lori CHUCK",
    "displayIndex": 4
  },
  "route": "/in-gc/hindi-chini-bhai-bhai"
}

$ cat /tmp/my.data.json| jq ".content.book1.name"
"giga"

$ cat /tmp/my.data.json| jq ".content.book1.field1"
"value1"

$ cat /tmp/my.data.json| jq ".content.book1.field-2"
jq: error (at <stdin>:27): null (null) and number (2) cannot be subtracted

$ cat /tmp/my.data.json| jq ".content.book1.field-three"
jq: error: three/0 is not defined at <top-level>, line 1:
.content.book1.field-three
jq: 1 compile error

$ cat /tmp/my.data.json| jq ".content.book1.field-three.url"
jq: error: three/0 is not defined at <top-level>, line 1:
.content.book1.field-three.url
jq: 1 compile error

$ cat /tmp/my.data.json| jq ".content.book1.field\-2"       
jq: error: syntax error, unexpected INVALID_CHARACTER, expecting $end (Unix shell quoting issues?) at <top-level>, line 1:
.content.book1.field\-2                    
jq: 1 compile error

$ cat /tmp/my.data.json| jq ".content.book1.field\\-2"
jq: error: syntax error, unexpected INVALID_CHARACTER, expecting $end (Unix shell quoting issues?) at <top-level>, line 1:
.content.book1.field\-2                    
jq: 1 compile error

$ cat /tmp/my.data.json| jq ".content.book1.'field-2'"
jq: error: syntax error, unexpected INVALID_CHARACTER, expecting FORMAT or QQSTRING_START (Unix shell quoting issues?) at <top-level>, line 1:
.content.book1.'field-2'               
jq: 1 compile error

$ cat /tmp/my.data.json| jq ".content.book1.authur"
{
  "name": "lori CHUCK",
  "displayIndex": 4
}

$ cat /tmp/my.data.json| jq ".content.book1.route"
"/in-gc/hindi-chini-bhai-bhai"

$

PS: I already know egrep so that's not what I'm looking for.

cat /tmp/my.data.json| jq ".content.book1"|egrep "short-url|field-2"
  "field-2": "value-2",
    "short-url": "book1/field-three/chota-chetan"

and someone really did a great job here: https://jqplay.org/

Answer 1

"-" is used for negation in jq. For key names with special characters such as "-", one cannot use the simplified ".keyname" syntax. There are several alternatives, but the most robust is simply to use the form .["KEY NAME"], which can be abbreviated to ["KEY NAME"] when chained, e.g. .a["b-c"] is shorthand for .a | .["b-c"].

If in doubt, use the pipe explicitly.

For further information, please consult the jq manual and/or https://github.com/stedolan/jq/wiki/FAQ

Answer 2

I don't know about jq, but you put python in the tags so:

$ cat test.json | python -c "import sys, json; print(json.load(sys.stdin)['content']['book1']['field-three']['name'])"
THIRD

or without the pipe:

$ python -c "import json; print(json.load(open('test.json'))['content']['book1']['field-three']['name'])"

Answer 3

As explained in the jq manual, to handle keys with non-identifier characters like - you can use double quotes.

From the shell this is easiest if you use single quotes around your filter. For example, try the following commands:

cat /tmp/my.data.json | jq '.pages'
cat /tmp/my.data.json | jq '.pages.book1[0]'
cat /tmp/my.data.json | jq '.pages.book1[1]'
cat /tmp/my.data.json | jq '.content'
cat /tmp/my.data.json | jq '.content.book1'
cat /tmp/my.data.json | jq '.content.book1.name'
cat /tmp/my.data.json | jq '.content.book1.field1'
cat /tmp/my.data.json | jq '.content.book1."field-2"'
cat /tmp/my.data.json | jq '.content.book1."field-three"'
cat /tmp/my.data.json | jq '.content.book1."field-three".url'
cat /tmp/my.data.json | jq '.content.book1.authur'
cat /tmp/my.data.json | jq '.content.book1.route'

Answer 4

Hm..took some time but finally it seems like we need to DOUBLE QUOTE it and back slash just the double quote for any key name containing a - in it's name.

$ cat /tmp/my.data.json| jq ".content.book1.\"field-2\""                    
"value-2"

$ cat /tmp/my.data.json| jq ".content.book1.\"field-three\".\"url\""  
"book1/field-three/"

OR if you wrap everything in a single quotes ', then we do NOT need to backslash " double quotes but use double quotes for key names with - in their name.

$ cat /tmp/my.data.json| jq '.content.book1."field-three"."url"'  
"book1/field-three/"

Hope it helps! Took some help/hint from https://jqplay.org/

See this for more: https://github.com/stedolan/jq/issues/38#issuecomment-9770240