What are the methods to migrate millions of nodes and edges from 0.44 to 0.5?

谁都会走 提交于 2019-12-24 21:29:02

问题


I'm migrating the entire Titan graph database from 0.44 to 0.5. There are about 120 million nodes and 90 million edges that's gigabytes of data. I tried the GraphML format, but it didn't work.

Can you suggest methods to do the migration?


回答1:


At the size you are describing you would probably execute the most efficient migration by using Titan-Hadoop/Faunus. The general process would be to:

  1. Use Faunus 0.4.x to extract the data from your graph as GraphSON and store that in HDFS
  2. Use Titan-Hadoop 0.5.x to read the GraphSON and write back to your storage backend.

Make sure that you've created your schema in your target backend prior to executing step 2.

As an aside, GraphML is not a good format for a graph of this size - it's will take too long and require a lot of resources if it would work at all. You might wonder why you wouldn't use Sequence files if you are using Faunus/Titan Hadoop...the reason you can't in this case is because I believe that there were version differences between 0.4.x and 0.5.x with respect to the file format of Sequence files. In other words, 0.5.x can't read 0.4.x sequence files. GraphSON is readable by both versions so it makes for an ideal migration format.



来源:https://stackoverflow.com/questions/27793596/what-are-the-methods-to-migrate-millions-of-nodes-and-edges-from-0-44-to-0-5

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!