How do I do a bulk insert in mySQL using node.js

Question

How would one do a bulk insert into mySQL if using something like https://github.com/felixge/node-mysql

Answer 1

@Ragnar123 answer is correct, but I see a lot of people saying in the comments that it is not working. I had the same problem and it seems like you need to wrap your array in [] like this:

var pars = [
    [99, "1984-11-20", 1.1, 2.2, 200], 
    [98, "1984-11-20", 1.1, 2.2, 200], 
    [97, "1984-11-20", 1.1, 2.2, 200]
];

It needs to be passed like [pars] into the method.

Answer 2

Bulk inserts are possible by using nested array, see the github page

Nested arrays are turned into grouped lists (for bulk inserts), e.g. [['a', 'b'], ['c', 'd']] turns into ('a', 'b'), ('c', 'd')

You just insert a nested array of elements.

An example is given in here

var mysql = require('mysql');
var conn = mysql.createConnection({
    ...
});

var sql = "INSERT INTO Test (name, email, n) VALUES ?";
var values = [
    ['demian', 'demian@gmail.com', 1],
    ['john', 'john@gmail.com', 2],
    ['mark', 'mark@gmail.com', 3],
    ['pete', 'pete@gmail.com', 4]
];
conn.query(sql, [values], function(err) {
    if (err) throw err;
    conn.end();
});

Note: values is an array of arrays wrapped in an array

[ [ [...], [...], [...] ] ]

There is also a totally different node-msql package for bulk insertion

Answer 3

I was having similar problem. It was just inserting one from the list of arrays. It worked after making the below changes.

1. Passed [params] to the query method.
2. Changed the query from insert (a,b) into table1 values (?) ==> insert (a,b) into table1 values ? . ie. Removed the paranthesis around the question mark.

Hope this helps. I am using mysql npm.

Answer 4

If you want to insert object, use this:

    currentLogs = [
 { socket_id: 'Server', message: 'Socketio online', data: 'Port  3333', logged: '2014-05-14 14:41:11' },
 { socket_id: 'Server', message: 'Waiting for Pi to connect...', data: 'Port: 8082', logged: '2014-05-14 14:41:11' }
];

console.warn(currentLogs.map(logs=>[ logs.socket_id , logs.message , logs.data , logs.logged ]));

The output will be:

[
  [ 'Server', 'Socketio online', 'Port  3333', '2014-05-14 14:41:11' ],
  [
    'Server',
    'Waiting for Pi to connect...',
    'Port: 8082',
    '2014-05-14 14:41:11'
  ]
]

Also, please check the documentation to know more about the map function.

Answer 5

I ran into this today (mysql 2.16.0) and thought I'd share my solution:

const items = [
    {name: 'alpha', description: 'describes alpha', value: 1},
    ...
];

db.query(
    'INSERT INTO my_table (name, description, value) VALUES ?',
    [items.map(item => [item.name, item.description, item.value])],
    (error, results) => {...}
);

Answer 6

All props to Ragnar123 for his answer.

I just wanted to expand it after the question asked by Josh Harington to talk about inserted IDs.

These will be sequential. See this answer : Does a MySQL multi-row insert grab sequential autoincrement IDs?

Hence you can just do this (notice what I did with the result.insertId):

  var statement = 'INSERT INTO ?? (' + sKeys.join() + ') VALUES ?';
  var insertStatement = [tableName, values];
  var sql = db.connection.format(statement, insertStatement);
  db.connection.query(sql, function(err, result) {
    if (err) {
      return clb(err);
    }
    var rowIds = [];
    for (var i = result.insertId; i < result.insertId + result.affectedRows; i++) {
      rowIds.push(i);
    }
    for (var i in persistentObjects) {
      var persistentObject = persistentObjects[i];
      persistentObject[persistentObject.idAttributeName()] = rowIds[i];
    }
    clb(null, persistentObjects);
  });

(I pulled the values from an array of objects that I called persistentObjects.)

Hope this helps.