Read/Write NetworkX Graph Object

Question

I am trying to deal with a super-massive NetworkX Graph object with hundreds of millions of nodes. I\'d like to be able to write it to file as to not consume all my computer mem

Answer 1

First try pickle; it's designed to serialize arbitrary objects.

An example of creating a DiGraph and serializing to a file:

import pickle
import networkx as nx

dg = nx.DiGraph()
dg.add_edge('a','b')
dg.add_edge('a','c')
pickle.dump(dg, open('/tmp/graph.txt', 'w'))

An example of loading a DiGraph from a file:

import pickle
import networkx as nx

dg = pickle.load(open('/tmp/graph.txt'))
print dg.edges()

Output:

[('a', 'c'), ('a', 'b')]

If this isn't efficient enough, I would write your own routine to serialize:

1. edges and
2. nodes (in case a node is incident to no edges).

Note that using list comprehensions when possible may be much more efficient (instead of standard for loops).

If this is not efficient enough, I'd call a C++ routine from within Python: http://docs.python.org/extending/extending.html