How to read json payload of a large response?

问题

I am developing some code that will read a large json payload returned by the Export API of Mailchimp.

        var response = caller.PostAsync(endpoint, data).Result;

        var exportedData = response.Content.ReadAsStringAsync().Result;

I am currently using the Visual Studio 2017 debugger to try and view the response but I keep getting an error that says:

Cannot obtain value of the local variable or argument because there is not enough memory available.

I ultimately want to extract information from each 'row' of this json data and do operations on them. What do I need to do so that I can view this data without running out of memory?

回答1:

A general approach to operating over large JSON files:

I'd start by avoiding any attempt to decode to string. Instead, grab the content stream of your response, following what is described in my answer here to avoid downloading the entire stream into memory before you get your hands on it.

Now you can:

var stream = await response.Content.ReadAsStreamAsync();

to grab a Stream instead of a string.

Now you can feed the stream directly into a deserializer, according to the information here:

using (StreamReader sr = new StreamReader(stream))
using (JsonReader reader = new JsonTextReader(sr))
{
    JsonSerializer serializer = new JsonSerializer();
    var o = serializer.Deserialize<SomeClassMatchingTheJsonSchema>(reader);
}

If you're still choking on memory, you might need to consider operating at a lower-level using JsonTextReader

回答2:

try

var exportedData = await response.Content.ReadAsAsync<model>();

also make sure your method is async

public async Task<returntype> MethodName()

let "model", in ReadAsAsync, be the name of the object model you're going to map the data to and let "returntype", next to Task in the method name, be whatever datatype your method is returning

can you update your question and show the entire method. I would like to take a closer look at it

来源：https://stackoverflow.com/questions/56398881/how-to-read-json-payload-of-a-large-response

标签

json

mailchimp

dotnet-httpclient