Need strategy advice for migrating large tables from RDS to DynamoDB

让人想犯罪 __ 提交于 2019-12-01 06:44:41

问题


We have a couple of mySql tables in RDS that are huge (over 700 GB), that we'd like to migrate to a DynamoDB table. Can you suggest a strategy, or a direction to do this in a clean, parallelized way? Perhaps using EMR or the AWS Data Pipeline.


回答1:


You can use AWS Pipeline. There are two basic templates, one for moving RDS tables to S3 and the second for importing data from S3 to DynamoDB. You can create your own pipeline using both templates.

Regards




回答2:


one thing to consider with such large data is whether Dynamo is the best option.

If this is statistical data or otherwise "big data", check out AWS RedShift which might be better suited for your situation.




回答3:


We have done a similar work and there is probably a better strategy to do this. Using AWS DMS and some prep tables within your source instance.

It involved two steps:

  1. You create new tables within your source instance which match exactly with the dynamodb schema. Like merging multiple tables to one etc.

  2. Set up DMS task with the prep tables as source and DynamoDB as the target. Since the prep tables and the target schema matches now, it should be pretty straightforward from this point.

Hope that helps! Good luck!!



来源:https://stackoverflow.com/questions/36212957/need-strategy-advice-for-migrating-large-tables-from-rds-to-dynamodb

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!