Import large csv file to mysql database using php

Question

I have a very large CSV file (150 MB). What is the best way to import it to MySQL? I have to do some manipulation in PHP before inserting it into the MySQL table.

Answer 1

I would just open it with fopen and use fgetcsv to read each line into an array. pseudo-php follows:

mysql_connect( //connect to db);

$filehandle = fopen("/path/to/file.csv", "r");
while (($data = fgetcsv($filehandle, 1000, ",")) !== FALSE) {
    // $data is an array
    // do your parsing here and insert into table
}

fclose($filehandle)

Answer 2

You could take a look at LOAD DATA INFILE in MySQL.

You might be able to do the manipulations once the data is loaded into MySQL, rather than first reading it into PHP. First store the raw data in a temporary table using LOAD DATA INFILE, then transform the data to the target table using a statement like the following:

INSERT INTO targettable (x, y, z)
SELECT foo(x), bar(y), z
FROM temptable