Jackson SAX parser throws exception when parsing huge JSON

问题

I'm trying to implement JSON array iterator with Jackson SAX parser (please, don't ask why). My app should work with huge files (up to 5 MiB), and that's a problem.

That's how I initialize JsonParser and call iterator creation. I create InputStream initialized with JSON placed in \raw folder.

    private JsonArrayIterator getIterator(String needle) throws IOException
    {
        InputStream inputStream = getApplicationContext().getResources().openRawResource(R.raw.products);
        inputStream.mark(-1);

        try {
        JsonParser jsonParser = createJsonParser(inputStream);

        // Some unrelated code

        return new JsonArrayIterator(jsonParser);
    } catch (IOException e) {
        e.printStackTrace();
    } finally {
        inputStream.close();
    }

        return null;
    }

And here is my iterator class.


    private Object parseNextObject() throws IOException {
        // I'm not using ObjectMapper because of reasons
        HashMap nextObject = new HashMap();

        int objectsCount = 1;

        while (objectsCount > 0) {
            JsonToken currentToken = currentParser.nextValue();

            if(currentToken == JsonToken.START_OBJECT) {
                ++objectsCount;
            }
            else if(currentToken == JsonToken.END_OBJECT) {
                --objectsCount;
            }
            else if(currentToken == JsonToken.START_ARRAY) {
                String currentName = currentParser.getCurrentName();
                ArrayList list = new ArrayList(100);
                JsonArrayIterator it = new JsonArrayIterator(currentParser);

                while (it.hasNext()) {
                    list.add(it.next());
                }

                nextObject.put(currentName, list);
            }
            else {
                // Here exception is throwed
                nextObject.put(currentParser.getCurrentName(), currentParser.getText());
            }
        }

        currentParser.nextToken();  // Skip END_OBJECT
        return nextObject;
    }

It seem to work perfe... OH, WAIT.

I have 3 sections (named arrays) in some huge file. It successfully parses first (a tiny one, under 1000 bytes). But I can't parse next.

Next array has nested array with simple objects (like this):


    {
        "properties":[
            {
                "id":"1",
                "title":"\u0426\u0432\u0435\u0442",
                "values":[
                    {
                        "id":"1_2",
                        "title":"\u0427\u0435\u0440\u043d\u044b\u0439"
                    },
                    {
                        "id":"1_5005",
                        "title":"\u0417\u0435\u043b\u0435\u043d\u044b\u0439"
                    },
                    {
                        "id":"1_5006",
                        "title":"\u0421\u0435\u0440\u044b\u0439"
                    }
                ]
            }
        ]
    }

For one object in values I call nextObject.put. currentParser.getCurrentName() is runs successfully and returns correct string, but currentParser.getText() fails. It's not a JSON problem: it perfectly being mapped on iOS. It's not an object or iterator creation problem: I can delete that where parser throws exception, but it will fail in the same place.

Here is stack trace:


    05-15 21:57:28.617: ERROR/AndroidRuntime(14758): FATAL EXCEPTION: main
    java.lang.NullPointerException: asset
    at android.content.res.AssetManager.readAsset(Native Method)
    at android.content.res.AssetManager.access$700(AssetManager.java:36)
    at android.content.res.AssetManager$AssetInputStream.read(AssetManager.java:576)
    at com.fasterxml.jackson.core.json.UTF8StreamJsonParser.loadMore(UTF8StreamJsonParser.java:174)
    at com.fasterxml.jackson.core.base.ParserBase.loadMoreGuaranteed(ParserBase.java:425)
    at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._finishString2(UTF8StreamJsonParser.java:1930)
    at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._finishString(UTF8StreamJsonParser.java:1911)
    at com.fasterxml.jackson.core.json.UTF8StreamJsonParser.getText(UTF8StreamJsonParser.java:276)
    at ru.studiomobile.JsonArrayIterator.parseNextObject(JsonArrayIterator.java:57)
    at ru.studiomobile.JsonArrayIterator.next(JsonArrayIterator.java:73)
    at ru.studiomobile.JsonArrayIterator.parseNextObject(JsonArrayIterator.java:47)
    at ru.studiomobile.JsonArrayIterator.next(JsonArrayIterator.java:73)
    at ru.studiomobile.MainActivity$2.onClick(MainActivity.java:81)
    at android.view.View.performClick(View.java:3127)
    at android.view.View$PerformClick.run(View.java:12025)
    at android.os.Handler.handleCallback(Handler.java:587)
    at android.os.Handler.dispatchMessage(Handler.java:92)
    at android.os.Looper.loop(Looper.java:132)
    at android.app.ActivityThread.main(ActivityThread.java:4126)
    at java.lang.reflect.Method.invokeNative(Native Method)
    at java.lang.reflect.Method.invoke(Method.java:491)
    at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:844)
    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:602)
    at dalvik.system.NativeStart.main(Native Method)

I noticed that there are something called Utf8StreamJsonParser. It has a field named _inputEnd equal 4000 (why 4000?). When other field, _inputPtr, become bigger it throw an exception. How do I can handle it? I tried to use BufferedInputStream with predefined block size instead of InputStream but it had no effect.

Update

Info about some lines


    47: list.add(it.next());  
    73: return parseNextObject();  
    75: e.printStackTrace();

Nothing special.

回答1:

It was an obvious problem: I forgot that I close stream after creating parser...

来源：https://stackoverflow.com/questions/10612006/jackson-sax-parser-throws-exception-when-parsing-huge-json

标签

java

android

json

jackson