How can I import data to Mongodb from Json file using java

Question

I am struggling with importing data into Mongodb from a Json file.
I can do the same in command line by using mongoimport command.

Answer 1

public static void importCSV(String path) {

        try {
            List<Document> list = new ArrayList<>();
            MongoDatabase db = DbConnection.getDbConnection();
            db.createCollection("newCollection");
            MongoCollection<Document> collection = db.getCollection("newCollection");
            BufferedReader reader = new BufferedReader(new FileReader(path));
            String line;
            while ((line = reader.readLine()) != null) {
                String[] item = line.split(","); // csv file is "" separated
                String id = item[0]; // get the value in the csv assign keywords
                String first_name = item[1];
                String last_name = item[2];
                String address = item[3];
                String gender = item[4];
                String dob = item[5];
                Document document = new Document(); // create a document
                document.put("id", id); // data into the database
                document.put("first_name", first_name);
                document.put("last_name", last_name);
                document.put("address", address);
                document.put("gender", gender);
                document.put("dob", dob);
                list.add(document);
            }
            collection.insertMany(list);

        }catch (Exception e){
            System.out.println(e);
        }
    }

Answer 2

List<Document> jsonList = new ArrayList<Document>();
net.sf.json.JSONArray array = net.sf.json.JSONArray.fromObject(json);
for (Object object : array) {
    net.sf.json.JSONObject jsonStr = (net.sf.json.JSONObject)JSONSerializer.toJSON(object);
    Document jsnObject = Document.parse(jsonStr.toString()); 
    jsonList.add(jsnObject);
}
collection.insertMany(jsonList);

Answer 3

With 3.2 driver, if you have a mongo collection and a collection of json documents e.g:

MongoCollection<Document> collection = ...
List<String> jsons = ...

You can insert individually:

jsons.stream().map(Document::parse).forEach(collection::insertOne);

or bulk:

collection.insertMany(
        jsons.stream().map(Document::parse).collect(Collectors.toList())
); 

Answer 4

Suppose you can read the JSON string respectively. For example, you read the first JSON text

{ "test_id" : 1245362, "name" : "ganesh", "age" : "28", "Job" : 
   {"company name" : "company1", "designation" : "SSE" } 
}

and assign it to a variable (String json1), the next step is to parse it,

DBObject dbo = (DBObject) com.mongodb.util.JSON.parse(json1);

put all dbo into a list,

List<DBObject> list = new ArrayList<>();
list.add(dbo);

then save them into database:

new MongoClient().getDB("test").getCollection("collection").insert(list);

EDIT:

In the newest MongoDB Version you have to use Documents instead of DBObject, and the methods for adding the object look different now. Here's an updated example:

Imports are:

import com.mongodb.MongoClient;
import com.mongodb.client.MongoDatabase;
import org.bson.Document;

The code would like this (refering to the text above the EDIT):

Document doc = Document.parse(json1);
new MongoClient().getDataBase("db").getCollection("collection").insertOne(doc);

you can also do it the way with the list. but then you need

new MongoClient().getDataBase("db").getCollection("collection").insertMany(list);

But I think there is a problem with this solution. When you type:

db.collection.find()

in the mongo shell to get all objects in the collection, the result looks like the following:

{ "_id" : ObjectId("56a0d2ddbc7c512984be5d97"),
    "test_id" : 1245362, "name" : "ganesh", "age" : "28", "Job" :
        { "company name" : "company1", "designation" : "SSE" 
    }
}

which is not exactly the same as before.

Answer 5

Runtime r = Runtime.getRuntime();

Process p = null;

//dir is the path to where your mongoimport is.

File dir=new File("C:/Program Files/MongoDB/Server/3.2/bin");

//this line will open your shell in giving dir, the command for import is exactly same as you use mongoimport in command promote

p = r.exec("c:/windows/system32/cmd.exe /c mongoimport --db mydb --collection student --type csv --file student.csv --headerline" ,null,dir);

Answer 6

I just faced this issue today and solved it in another different way while none here satisfied me, so enjoy my extra contribution. Performances are sufficient to export 30k documents and import them in my Springboot app for integration test cases (takes a few seconds).

First, the way your export your data in the first place matters. I wanted a file where each line contains 1 document that I can parse in my java app.

mongo db --eval 'db.data.find({}).limit(30000).forEach(function(f){print(tojson(f, "", true))})' --quiet > dataset.json

Then I get the file from my resources folder, parse it, extract lines, and process them with mongoTemplate. Could use a buffer.

@Autowired    
private MongoTemplate mongoTemplate;

public void createDataSet(){
    mongoTemplate.dropCollection("data");
    try {
        InputStream inputStream = Thread.currentThread().getContextClassLoader().getResourceAsStream(DATASET_JSON);
        List<Document> documents = new ArrayList<>();
        String line;
        InputStreamReader isr = new InputStreamReader(inputStream, Charset.forName("UTF-8"));
        BufferedReader br = new BufferedReader(isr);
        while ((line = br.readLine()) != null) {
            documents.add(Document.parse(line));
        }
        mongoTemplate.insert(documents,"data");


    } catch (Exception e) {
        throw new RuntimeException(e);
    }
}