Slow Import of Large MySQL Dump

 ̄綄美尐妖づ 提交于 2019-12-11 01:19:00

问题


I'm trying to import a very large MySQL dump file into a new MySQL server. However, after a certain point, it seems to bog down. I'm about 250M rows in, and it's taking over 10-12 seconds to run even a single line of the dump. It seems to hang on "update".

Am I missing something I should do to make this import go faster? It's a pretty beefy server so I don't think it's the I/O..


回答1:


What seemed to work for me with a 'beefy' server, was splitting the file into several smaller and importing them all in parallel. Altough that may break your row ordering. (and results may vary depending on storage engine)

IIRC there was some kind of script to do that, but it was unmaintained. So when I was messing with that, I used just head/tail and pipes.

If you are inserting into multiple tables and not using foreign keys, i could probably provide you my simple perl script that I use to split one big dump into per table dumps, which can in turn be imported in parallel.

Also this may help. Either done manually (insert disable at the beginning and enable at the end of your dump file) or when dumping the db. This should speed up your inserts, but it will take quite some time to build indexes.




回答2:


If you're sure that your data is correct, try disabling foreign checks so the process can be speed up:

SET @@FOREIGN_KEY_CHECKS = 0;


来源:https://stackoverflow.com/questions/6863024/slow-import-of-large-mysql-dump

标签
易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!