How can I completely sort arbitrary JSON using jq?

Question

I want to diff two JSON text files. Unfortunately they\'re constructed in arbitrary order, so I get diffs when they\'re semantically identical. I\'d like to use jq (or whateve

Answer 1

Here is a solution using a generic function sorted_walk/1 (so named for the reason described in the postscript below).

normalize.jq:

# Apply f to composite entities recursively using keys[], and to atoms
def sorted_walk(f):
  . as $in
  | if type == "object" then
      reduce keys[] as $key
        ( {}; . + { ($key):  ($in[$key] | sorted_walk(f)) } ) | f
  elif type == "array" then map( sorted_walk(f) ) | f
  else f
  end;

def normalize: sorted_walk(if type == "array" then sort else . end);

normalize

Example using bash:

diff <(jq -S -f normalize.jq FILE1) <(jq -S -f normalize.jq FILE2)

POSTSCRIPT: The builtin definition of walk/1 was revised after this response was first posted: it now uses keys_unsorted rather than keys.

Answer 2

I want to diff two JSON text files.

Use jd with the -set option:

No output means no difference.

$ jd -set A.json B.json

Differences are shown as an @ path and + or -.

$ jd -set A.json C.json

@ ["People",{}]
+ "Carla"

The output diffs can also be used as patch files with the -p option.

$ jd -set -o patch A.json C.json; jd -set -p patch B.json

{"City":"Boston","People":["John","Carla","Bryan"],"State":"MA"}

https://github.com/josephburnett/jd#command-line-usage