Tuning Neo4j for Performance

Question

I have imported data using Michael Hunger\'s Batch Import, through which I created:-

4,612,893 nodes
14,495,063 properties
    node properties are indexed.
5,300         


        

Answer 1

Okay, so first of all, for only 8GB of memory that's a very large graph. You should seriously consider getting a larger box. Neo4j actually provides an extremely nice hardware calculator that will let you determine exactly what is appropriate for your needs:

http://neotechnology.com/calculatorv2/

In a contrived way (since there are more relevant metrics to determining size) their calculator estimates should should be dedicating about 10GB at a minimum.

Secondly, Neo4j and any graph database will have issues with nodes that have a large number of connections. If you're looking to tune your instance to perform better (after getting a bigger box) I would suggest looking for any massive nodes with a large number of connections as those will seriously impact performance.

After seeing your examples I'm quite certain you've got a graph with a number of nodes that have a much larger number of connections that other nodes. This will inherently slow down your performance. You might also try more narrow queries. Especially when you're already working on a server that's too small you don't want to run the kind of extremely taxing large return queries you've got there.

There are some things about your queries that could be cleaned up, but I really urge you to get the appropriately sized box for your graph and actually do some introspection into the number of connections your most connected nodes have.

It also looks like you have an artificial cap on your Java Heap size. If you try starting up java with a command like:

java -Xmx8g //Other stuff

You'll allocate 8 gigs instead of the standard ~500 Megs, which would also help.