How to deserialize JSON into flat, Map-like structure?

Question

Have in mind that the JSON structure is not known before hand i.e. it is completely arbitrary, we only know that it is JSON format.

For example,

The followin

Answer 1

You can do this to traverse the tree and keep track of how deep you are to figure out dot notation property names:

import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.node.ArrayNode;
import com.fasterxml.jackson.databind.node.ObjectNode;
import com.fasterxml.jackson.databind.node.ValueNode;
import java.io.IOException;
import java.util.HashMap;
import java.util.Iterator;
import java.util.Map;
import org.junit.Test;

public class FlattenJson {
  String json = "{\n" +
      "   \"Port\":\n" +
      "   {\n" +
      "       \"@alias\": \"defaultHttp\",\n" +
      "       \"Enabled\": \"true\",\n" +
      "       \"Number\": \"10092\",\n" +
      "       \"Protocol\": \"http\",\n" +
      "       \"KeepAliveTimeout\": \"20000\",\n" +
      "       \"ThreadPool\":\n" +
      "       {\n" +
      "           \"@enabled\": \"false\",\n" +
      "           \"Max\": \"150\",\n" +
      "           \"ThreadPriority\": \"5\"\n" +
      "       },\n" +
      "       \"ExtendedProperties\":\n" +
      "       {\n" +
      "           \"Property\":\n" +
      "           [                         \n" +
      "               {\n" +
      "                   \"@name\": \"connectionTimeout\",\n" +
      "                   \"$\": \"20000\"\n" +
      "               }\n" +
      "           ]\n" +
      "       }\n" +
      "   }\n" +
      "}";

  @Test
  public void testCreatingKeyValues() {
    Map<String, String> map = new HashMap<String, String>();
    try {
      addKeys("", new ObjectMapper().readTree(json), map);
    } catch (IOException e) {
      e.printStackTrace();
    }
    System.out.println(map);
  }

  private void addKeys(String currentPath, JsonNode jsonNode, Map<String, String> map) {
    if (jsonNode.isObject()) {
      ObjectNode objectNode = (ObjectNode) jsonNode;
      Iterator<Map.Entry<String, JsonNode>> iter = objectNode.fields();
      String pathPrefix = currentPath.isEmpty() ? "" : currentPath + ".";

      while (iter.hasNext()) {
        Map.Entry<String, JsonNode> entry = iter.next();
        addKeys(pathPrefix + entry.getKey(), entry.getValue(), map);
      }
    } else if (jsonNode.isArray()) {
      ArrayNode arrayNode = (ArrayNode) jsonNode;
      for (int i = 0; i < arrayNode.size(); i++) {
        addKeys(currentPath + "[" + i + "]", arrayNode.get(i), map);
      }
    } else if (jsonNode.isValueNode()) {
      ValueNode valueNode = (ValueNode) jsonNode;
      map.put(currentPath, valueNode.asText());
    }
  }
}

It produces the following map:

Port.ThreadPool.Max=150, 
Port.ThreadPool.@enabled=false, 
Port.Number=10092, 
Port.ExtendedProperties.Property[0].@name=connectionTimeout, 
Port.ThreadPool.ThreadPriority=5, 
Port.Protocol=http, 
Port.KeepAliveTimeout=20000, 
Port.ExtendedProperties.Property[0].$=20000, 
Port.@alias=defaultHttp, 
Port.Enabled=true

It should be easy enough to strip out @ and $ in the property names, although you could end up with collisions in key names since you said the JSON was arbitrary.

Answer 2

How about using the json-flattener. https://github.com/wnameless/json-flattener

BTW, I am the author of this lib.

String flattenedJson = JsonFlattener.flatten(yourJson);
Map<String, Object> flattenedJsonMap = JsonFlattener.flattenAsMap(yourJson);

// Result:
{
    "Port.@alias":"defaultHttp",
    "Port.Enabled":"true",
    "Port.Number":"10092",
    "Port.Protocol":"http",
    "Port.KeepAliveTimeout":"20000",
    "Port.ThreadPool.@enabled":"false",
    "Port.ThreadPool.Max":"150",
    "Port.ThreadPool.ThreadPriority":"5",
    "Port.ExtendedProperties.Property[0].@name":"connectionTimeout",
    "Port.ExtendedProperties.Property[0].$":"20000"
}

Answer 3

If you know the structure beforehand, you can define a Java class and use gson to parse JSON into an instance of that class:

YourClass obj = gson.fromJson(json, YourClass.class); 

If not, then I'm not sure what you're trying to do. You obviously can't define a class on-the-fly so accessing the parsed JSON using dot-notation is out of the question.

Unless you want something like:

Map<String, String> parsed = magicParse(json);
parsed["Port.ThreadPool.max"]; // returns 150

If so, then traversing your map of maps and building a "flattened" map doesn't seem too much of a problem.

Or is it something else?

Answer 4

org.springframework.integration.transformer.ObjectToMapTransformer from Spring Integration produces desired result. By default it has shouldFlattenKeys property set to true and produces flat maps (no nesting, value is always simple type). When shouldFlattenKeys=false it produces nested maps

ObjectToMapTransformer is meant to be used as part of integration flow, but it is perfectly fine to use it in stand-alone way. You need to construct org.springframework.messaging.Message with payload of transformation input. transform method returns org.springframework.messaging.Message object with payload that is Map

import org.springframework.integration.transformer.ObjectToMapTransformer;
import org.springframework.messaging.Message;
import org.springframework.messaging.support.GenericMessage;

Message message = new GenericMessage(value);
 ObjectToMapTransformer transformer = new ObjectToMapTransformer();
        transformer.setShouldFlattenKeys(true);
        Map<String,Object> payload = (Map<String, Object>) transformer
                .transform(message)
                .getPayload();

Side note: It is probably overkill to add Spring Integration to the classpath just to use single class, but you may check implementation of this class and write similar solution on your own. Nested map is produced by Jackson (org.springframework.integration.support.json.JsonObjectMapper#fromJson(payload, Map.class)), then mapis travered recursively, flattening all values that are collections.

Answer 5

You can achieve something like that using the Typesafe Config Library as in the following example:

import com.typesafe.config.*;
import java.util.Map;
public class TypesafeConfigExample {
  public static void main(String[] args) {
    Config cfg = ConfigFactory.parseString(
      "   \"Port\":\n" +
      "   {\n" +
      "       \"@alias\": \"defaultHttp\",\n" +
      "       \"Enabled\": \"true\",\n" +
      "       \"Number\": \"10092\",\n" +
      "       \"Protocol\": \"http\",\n" +
      "       \"KeepAliveTimeout\": \"20000\",\n" +
      "       \"ThreadPool\":\n" +
      "       {\n" +
      "           \"@enabled\": \"false\",\n" +
      "           \"Max\": \"150\",\n" +
      "           \"ThreadPriority\": \"5\"\n" +
      "       },\n" +
      "       \"ExtendedProperties\":\n" +
      "       {\n" +
      "           \"Property\":\n" +
      "           [                         \n" +
      "               {\n" +
      "                   \"@name\": \"connectionTimeout\",\n" +
      "                   \"$\": \"20000\"\n" +
      "               }\n" +
      "           ]\n" +
      "       }\n" +
      "   }\n" +
      "}");

    // each key has a similar form to what you need
    for (Map.Entry<String, ConfigValue> e : cfg.entrySet()) {
      System.out.println(e);
    }
  }
}

Answer 6

import com.alibaba.fastjson.JSONObject;
import com.alibaba.fastjson.JSONPath;

import java.util.Iterator;
import java.util.Map;

public class JsonMapFlattener {


    public static Map<String, Object> flatten(Map<String, ? extends Object> inputMap) {
        return org.springframework.vault.support.JsonMapFlattener.flatten(inputMap);
    }


    public static JSONObject unflatten(Map<String, ? extends Object> inputMap) {
        Map<String, Object> map = flatten(inputMap);
        Iterator<String> it = map.keySet().iterator();
        JSONObject jo2 = new JSONObject();
        while (it.hasNext()) {
            String key = it.next();
            Object value = map.get(key);
            JSONPath.set(jo2, key, value);
        }
        return jo2;

    }
}