Import more than 1 json file using mongoimport

Question

I am new to mongodb and want to know about importing a json file from one server to another. I tried the following command mongoimport -d tes

Answer 1

You can always write some shell scripts.

colls=( mycoll1 mycoll2 mycoll5 )

for c in ${colls[@]}
do
  mongoimport -d mydb -c $c.json
done

Answer 2

I used the solutions here to add a shell function to my bash profile for doing this quickly.

My example depends on the mongo export outputting each collection as a file with the collection name and .metadata.json extension.

function mimport() { for filename in *; do collection="${filename%.metadata.json}"; mongoimport --db $1 --collection $collection --file $filename; done }

Use in the path of the export files, passing the DB name to the command...

mimport my_db

Will load all collections into the DB at localhost.

Answer 3

Not sure whether it's a new feature, but mongoimport now can actually read from stdin. So what one can do to import multiple JSON files is as simple as

cat *.json | mongoimport --uri "mongdb://user:password@host/db?option=value" --collection example

I'm using mongodb-tools v4.2.0 btw.

UPDATE

mongodbimport can potentially consume a high amount of memory which may cause the program to be kill by system OOM. My machine's got 32GB RAM and this happened consistently when I tried to import ~10GB of data which is stored in RAM disk.

To divide a relatively large job into batches:

#!/usr/bin/env bash

declare -a json_files=()
for f in *.json; do
    json_files+="$f"
    if [[ "${#json_files[@]}" -ge 1000 ]]; then
        cat "${json_files[@]}" | mongoimport --uri="mongodb://user:pass@host/db" --collection=examples -j8 #--mode=upsert --upsertFields=id1
        json_files=()
    fi
done

Answer 4

For windows bat file. This would be way better if you have a list of json files in the folder. and the collection name matches the name in files

@echo off
for %%f in (*.json) do (
    "mongoimport.exe" --db databasename --collection %%~nf --drop --file %%f
)
pause

Answer 5

Windows Batch version:

@echo off
for %%f in (*.json) do (
    "mongoimport.exe" --jsonArray --db databasename --collection collectioname --file %%~nf.json
)

Answer 6

One line solution:

for /F %i in ('dir /b c:\files\*.json') do mongoimport.exe /d db /c files /file c:\file\%i