Including DB function calls in python MySQLdb executemany()

Question

When I try to run statements like:

cursor.executemany("""INSERT INTO `test` (`id`,`data`,`time_added`) 
                      VALUES (%s, %s, NOW())""", [(i.id, i.data) for i in items])

MySQLdb seems to choke on the ) in NOW(), when it expands the list of rows to be inserted because it sees that parenthesis as the end of the value block. That is, the queries look like:

('1', 'a', NOW(), ('2','b', NOW(), ('3','c',NOW())

And MYSQL reports a syntax error. Instead, they should look like:

('1', 'a', NOW()), ('2','b', NOW()), ('3','c',NOW())

There should be some way to escape the NOW(), but I can't figure out how. Adding 'NOW()' to the tuple doesn't work because then NOW() is quoted and interpreted by the DB as a string rather than a function call.

Working around this by using current timestamp as default value is not an option -- this is an example, I need to do this sort of thing with a variety of db functions, not just now.

Thanks!

Answer 1

The method below is far from ideal, but, unfortunately, it is the only way I know.

The idea is to manually construct the SQL, using connection.literal to escape the arguments for you:

cursor=connection.cursor()
args=[(1,'foo'),(2,'bar')]
sql=('INSERT INTO `foo` (`fooid`,`data`,`time_added`) VALUES '
     +','.join(
         ['(%s,%s,NOW())'%connection.literal(arg)
          for arg in args]))
cursor.execute(sql)

This looks horrible, and may make your skin crawl, but if you look under the hood (in /usr/lib/pymodules/python2.6/MySQLdb/cursors.py) at what MySQLdb is doing in cursors.executemany, I think this is along the same lines as what that function is doing, minus the mixup due the regex cursors.insert_values not correctly parsing the nested parentheses. (eek!)

---

I've just installed oursql, an alternative to MySQLdb, and am happy to report that

sql='INSERT INTO `foo` (`fooid`,`data`,`time_added`) VALUES (?,?,NOW())'
cursor.executemany(sql,args)

works as expected with oursql.