Split huge (95Mb) JSON array into smaller chunks?

Question

I exported some data from my database in the form of JSON, which is essentially just one [list] with a bunch (900K) of {objects} inside it.

Trying to import it on my

Answer 1

I know this is question is from a while back, but I think this new solution is hassle-free.

You can use pandas 0.21.0 which supports a chunksize parameter as part of read_json. You can load one chunk at a time and save the json:

import pandas as pd
chunks = pd.read_json('file.json', lines=True, chunksize = 20)
for i, c in enumerate(chunks):
    c.to_json('chunk_{}.json'.format(i))