How to do an initial batch import of CSV / MySQL data into neo4j database

Question

I am considering replacing a MySQL database with a neo4j database. I am a complete beginner with neo4j and would like to know how to go about doing a batch insert of my current

Answer 1

Based on advice in the git repo. Using Michael Hunger's batch-import it is possible to import multiple node types from the one .csv file. To quote Michael:

Just put them all into one nodes file, you can have any attribute not having a value in a certain row, it will then just be skipped.

So the general approach i used was:

combine all the nodes tables into a new table called nodes:

1. Create a new table nodes with an auto incrementing newID field and a type field. the type field will record what table the node data came from
2. Add all the possible columns names from the 3 node tables allowing nulls.
3. INSERT INTO nodes the values from Person, then Organism, then Story, in addition to setting the type field to person, organism, or story. Leave any unrelated fields blank.

in another new table rels add the newly created newID indexes to the Links table based on a sql JOIN:

INSERT INTO rels
SELECT  
    n1.newID AS fromNodeID, 
    n2.newID AS toNodeID,
    L.LinkType,
    L.ID
FROM 
    Links L
LEFT JOIN 
    nodes n1 
    ON 
    L.fromID = n1.ID 
    AND 
    L.fromType = n1.type
LEFT JOIN 
    nodes n2 
    ON 
    L.toID = n2.ID 
    AND 
    L.toType = n2.type;

Then export these two new tables nodes and rels as Tab seperated .csv files, and use them with batch-import:

$java -server -Xmx4G -jar target/batch-import-jar-with-dependencies.jar target/graph.db nodes.csv rels.csv

Answer 2

As you say that you are happy working with python and shell scripts, you may also want to have a look at the command line tools which come with py2neo, in particular geoff. This uses a flat file format I put together for holding graph data so in your instance, you would need to build a flat file from your source data and insert this into your graph database.

The file format and server plugin are documented here and the py2neo module for the client application is here.

If anything is missing from the docs or you want more information about this then feel free to drop me an email

Nigel