Flatten nested JSON using jq

Question

I\'d like to flatten a nested json object, e.g. {\"a\":{\"b\":1}} to {\"a.b\":1} in order to digest it in solr.

I have 11 TB of json files whi

Answer 1

This is just a variant of Santiago's jq:

. as $in 
| reduce leaf_paths as $path ({};
     . + { ($path | map(tostring) | join(".")): $in | getpath($path) })

It avoids the overhead of the key/value construction and destruction.

(If you have access to a version of jq later than jq 1.5, you can omit the "map(tostring)".)

Two important points about both these jq solutions:

1. Arrays are also flattened. E.g. given {"a": {"b": [0,1,2]}} as input, the output would be:

   {
     "a.b.0": 0,
     "a.b.1": 1,
     "a.b.2": 2
   }
   

2. If any of the keys in the original JSON contain periods, then key collisions are possible; such collisions will generally result in the loss of a value. This would happen, for example, with the following input:

   {"a.b":0, "a": {"b": 1}}