How do I read a data from a JSON file with high efficiency in Java with Jackson?

Question

I store all static data in the JSON file. This JSON file has up to 1000 rows. How to get the desired data without storing all rows as

Answer 1

You can use DSM stream parsing library for memory, CPU efficiency and fast development. DSM uses YAML based mapping file and reads the whole data only once.

Here is the solution of your question:

Mapping File:

params:
   colorsToFilter: ['Blue','Red']  # parameteres can be passed programmatically
result:
   type: array
   path: /.*colors  # path is regex
   filter: params.colorsToFilter.contains(self.data.color)  # select only color that exist in colorsToFilter list
   fields:
      color: 
      code:
         type: array

Usage of DSM to parse json:

DSM dsm = new DSMBuilder(new File("path/maping.yaml")).create(Colors.class);
List<Colors> object = (List<Colors>) dsm.toObject(jsonData);

System.out.println(object);

Output:

[Colors{color='Blue', code=[012, 0324, 15478, 7412]}, Colors{color='Red', code=[145, 001, 1, 7879, 123984, 89]}]

Answer 2

Creating POJO classes in this case is a wasting because we do not use the whole result List<Colors> but only one internal property. To avoid this we can use native JsonNode and ArrayNode data types. We can read JSON using readTree method, iterate over array, find given object and finally convert internal code array. It could look like below:

import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.node.ArrayNode;

import java.io.File;
import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;

public class JsonApp {

    public static void main(String[] args) throws Exception {
        File jsonFile = new File("./resource/test.json").getAbsoluteFile();

        ObjectMapper mapper = new ObjectMapper();

        ArrayNode rootArray = (ArrayNode) mapper.readTree(jsonFile);
        int size = rootArray.size();

        for (int i = 0; i < size; i++) {
            JsonNode jsonNode = rootArray.get(i);
            if (jsonNode.get("color").asText().equals("Blue")) {
                Iterator<JsonNode> codesIterator = jsonNode.get("code").elements();
                List<String> codes = new ArrayList<>();
                codesIterator.forEachRemaining(n -> codes.add(n.asText()));

                System.out.println(codes);
                break;
            }
        }
    }
}

Above code prints:

[012, 0324, 15478, 7412]

Downside of this solution is we load the whole JSON to memory which could be a problem for us. Let's try to use Streaming API to do that. It is a bit difficult to use and you must know how your JSON payload is constructed but it is the fastest way to get code array using Jackson. Below implementation is naive and does not handle all possibilities so you should not rely on it:

import com.fasterxml.jackson.core.JsonFactory;
import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.core.JsonToken;

import java.io.File;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;

public class JsonApp {

    public static void main(String[] args) throws Exception {
        File jsonFile = new File("./resource/test.json").getAbsoluteFile();

        System.out.println(getBlueCodes(jsonFile));
    }

    private static List<String> getBlueCodes(File jsonFile) throws IOException {
        try (JsonParser parser = new JsonFactory().createParser(jsonFile)) {
            while (parser.nextToken() != JsonToken.END_OBJECT) {
                String fieldName = parser.getCurrentName();
                // Find color property
                if ("color".equals(fieldName)) {
                    parser.nextToken();
                    // Find Blue color
                    if (parser.getText().equals("Blue")) {
                        // skip everything until start of the array
                        while (parser.nextToken() != JsonToken.START_ARRAY) ;

                        List<String> codes = new ArrayList<>();
                        while (parser.nextToken() != JsonToken.END_ARRAY) {
                            codes.add(parser.getText());
                        }
                        return codes;
                    } else {
                        // skip current object because it is not `Blue`
                        while (parser.nextToken() != JsonToken.END_OBJECT) ;
                    }
                }
            }
        }

        return Collections.emptyList();
    }
}

Above code prints:

[012, 0324, 15478, 7412]

At the end I need to mention about JsonPath solution which also can be good if you can use other library:

import com.jayway.jsonpath.JsonPath;
import net.minidev.json.JSONArray;

import java.io.File;
import java.util.List;
import java.util.stream.Collectors;

public class JsonPathApp {
    public static void main(String[] args) throws Exception {
        File jsonFile = new File("./resource/test.json").getAbsoluteFile();

        JSONArray array = JsonPath.read(jsonFile, "$[?(@.color == 'Blue')].code");
        JSONArray jsonCodes = (JSONArray)array.get(0);
        List<String> codes = jsonCodes.stream()
                .map(Object::toString).collect(Collectors.toList());

        System.out.println(codes);
    }
}

Above code prints:

[012, 0324, 15478, 7412]