Inserting data from CSV into MySQL DB is very slow

Question

Trying to insert data from a CSV file to a MySQL DB using Ruby, and it\'s very slow. Note that this is not a Rails application, just stand-alone Ruby script.

Here is my

Answer 1

Use the zdennis/activerecord-import gem. you can insert tons records quickly.

Answer 2

Use Load Data Infile.

Here is a nice article on performance and strategies titled Testing the Fastest Way to Import a Table into MySQL. Don't let the mysql version of the title or inside the article scare you away. Jumping to the bottom and picking up some conclusions:

The fastest way you can import a table into MySQL without using raw files is the LOAD DATA syntax. Use parallelization for InnoDB for better results, and remember to tune basic parameters like your transaction log size and buffer pool. Careful programming and importing can make a >2-hour problem became a 2-minute process. You can disable temporarily some security features for extra performance

You might just find your times greatly reduced.