PDO Prepared Inserts multiple rows in single query

Question

I am currently using this type of SQL on MySQL to insert multiple rows of values in one single query:

INSERT INTO `tbl` (`key1`,`key2`) VALUES (\'r1v1\',\'r1         


        

Answer 1

Based on my experiments I found out that mysql insert statement with multiple value rows in single transaction is the fastest one.

However, if the data is too much then mysql's max_allowed_packet setting might restrict the single transaction insert with multiple value rows. Hence, following functions will fail when there is data greater than mysql's max_allowed_packet size:

1. singleTransactionInsertWithRollback
2. singleTransactionInsertWithPlaceholders
3. singleTransactionInsert

The most successful one in insert huge data scenario is transactionSpeed method, but it consumes time more the above mentioned methods. So, to handle this problem you can either split your data into smaller chunks and call single transaction insert multiple times or give up speed of execution by using transactionSpeed method.

Here's my research

data = [];
        $this->pdo = new \PDO('mysql:dbname=test_data', 'admin', 'admin');
        if (!$this->pdo) {
            die('Failed to connect to database');
        }
    }

    public function createData()
    {
        $prefix = 'test';
        $postfix = 'unicourt.com';
        $salutations = ['Mr.', 'Ms.', 'Dr.', 'Mrs.'];

        $csv[] = ['Salutation', 'First Name', 'Last Name', 'Email Address'];
        for ($i = 0; $i < 100000; ++$i) {
            $csv[] = [
                $salutations[$i % \count($salutations)],
                $prefix.$i,
                $prefix.$i,
                $prefix.$i.'@'.$postfix,
            ];
        }

        $this->data = $csv;
    }

    public function truncateTable()
    {
        $this->pdo->query('TRUNCATE TABLE `name`');
    }

    public function transactionSpeed()
    {
        $timer1 = microtime(true);
        $this->pdo->beginTransaction();
        $sql = 'INSERT INTO `name` (`first_name`, `last_name`) VALUES (:first_name, :last_name)';
        $sth = $this->pdo->prepare($sql);

        foreach (\array_slice($this->data, 1) as $values) {
            $sth->execute([
                ':first_name' => $values[1],
                ':last_name' => $values[2],
            ]);
        }

        // $timer2 = microtime(true);
        // echo 'Prepare Time: '.($timer2 - $timer1).PHP_EOL;
        // $timer3 = microtime(true);

        if (!$this->pdo->commit()) {
            echo "Commit failed\n";
        }
        $timer4 = microtime(true);
        // echo 'Commit Time: '.($timer4 - $timer3).PHP_EOL;

        return $timer4 - $timer1;
    }

    public function autoCommitSpeed()
    {
        $timer1 = microtime(true);
        $sql = 'INSERT INTO `name` (`first_name`, `last_name`) VALUES (:first_name, :last_name)';
        $sth = $this->pdo->prepare($sql);
        foreach (\array_slice($this->data, 1) as $values) {
            $sth->execute([
                ':first_name' => $values[1],
                ':last_name' => $values[2],
            ]);
        }
        $timer2 = microtime(true);

        return $timer2 - $timer1;
    }

    public function noBindAutoCommitSpeed()
    {
        $timer1 = microtime(true);

        foreach (\array_slice($this->data, 1) as $values) {
            $sth = $this->pdo->prepare("INSERT INTO `name` (`first_name`, `last_name`) VALUES ('{$values[1]}', '{$values[2]}')");
            $sth->execute();
        }
        $timer2 = microtime(true);

        return $timer2 - $timer1;
    }

    public function singleTransactionInsert()
    {
        $timer1 = microtime(true);
        foreach (\array_slice($this->data, 1) as $values) {
            $arr[] = "('{$values[1]}', '{$values[2]}')";
        }
        $sth = $this->pdo->prepare('INSERT INTO `name` (`first_name`, `last_name`) VALUES '.implode(', ', $arr));
        $sth->execute();
        $timer2 = microtime(true);

        return $timer2 - $timer1;
    }

    public function singleTransactionInsertWithPlaceholders()
    {
        $placeholders = [];
        $timer1 = microtime(true);
        $sql = 'INSERT INTO `name` (`first_name`, `last_name`) VALUES ';
        foreach (\array_slice($this->data, 1) as $values) {
            $placeholders[] = '(?, ?)';
            $arr[] = $values[1];
            $arr[] = $values[2];
        }
        $sql .= implode(', ', $placeholders);
        $sth = $this->pdo->prepare($sql);
        $sth->execute($arr);
        $timer2 = microtime(true);

        return $timer2 - $timer1;
    }

    public function singleTransactionInsertWithRollback()
    {
        $placeholders = [];
        $timer1 = microtime(true);
        $sql = 'INSERT INTO `name` (`first_name`, `last_name`) VALUES ';
        foreach (\array_slice($this->data, 1) as $values) {
            $placeholders[] = '(?, ?)';
            $arr[] = $values[1];
            $arr[] = $values[2];
        }
        $sql .= implode(', ', $placeholders);
        $this->pdo->beginTransaction();
        $sth = $this->pdo->prepare($sql);
        $sth->execute($arr);
        $this->pdo->commit();
        $timer2 = microtime(true);

        return $timer2 - $timer1;
    }
}

$s = new SpeedTestClass();
$s->createData();
$s->truncateTable();
echo "Time Spent for singleTransactionInsertWithRollback: {$s->singleTransactionInsertWithRollback()}".PHP_EOL;
$s->truncateTable();
echo "Time Spent for single Transaction Insert: {$s->singleTransactionInsert()}".PHP_EOL;
$s->truncateTable();
echo "Time Spent for single Transaction Insert With Placeholders: {$s->singleTransactionInsertWithPlaceholders()}".PHP_EOL;
$s->truncateTable();
echo "Time Spent for transaction: {$s->transactionSpeed()}".PHP_EOL;
$s->truncateTable();
echo "Time Spent for AutoCommit: {$s->noBindAutoCommitSpeed()}".PHP_EOL;
$s->truncateTable();
echo "Time Spent for autocommit with bind: {$s->autoCommitSpeed()}".PHP_EOL;
$s->truncateTable();

The results for 100,000 entries for a table containing only two columns is as below

$ php data.php
Time Spent for singleTransactionInsertWithRollback: 0.75147604942322
Time Spent for single Transaction Insert: 0.67445182800293
Time Spent for single Transaction Insert With Placeholders: 0.71131205558777
Time Spent for transaction: 8.0056409835815
Time Spent for AutoCommit: 35.4979159832
Time Spent for autocommit with bind: 33.303519010544