Python2.7 MySQL Connector Error on LOAD DATA LOCAL INFILE

Question

I\'m trying to dynamically load census data into a mysql database (from .csv files) using Python and MySQL Connector.

I can\'t figure out why I am getting the error

Answer 1

Add field allow_local_infile = "True" when you do mysql.connector.connect. It'll work

Answer 2

Okay, here's what I've figured out.

@Dd_tch - your answer helped me realize that this code would work better:

query = add_csv_file % csv_info
cursor.execute(query)

Though the MySQL site for Connector seems to indicate you could do what I was doing (http://dev.mysql.com/doc/connector-python/en/connector-python-example-cursor-transaction.html), that wasn't working.

When I fixed that, I got a new error: mysql.connector.errors.ProgrammingError: 1148 (42000): The used command is not allowed with this MySQL version

There are several sites that indicate that this could be fixed by adding "local_infile=1" to the MySQL Connector config. Here's one: http://dev.mysql.com/doc/refman/5.1/en/loading-tables.html

This solution did not work for me. My local_infile is set to 1 on MySQL and I can't set it within the Python code or I get an e

I'm going to instead replace the LOAD LOCAL DATA INFILE with something that will read CSV files line by line and insert lines into the database. This will make the code more portable to other DBs anyway.

Answer 3

You have an error when calling the execute() method:

cursor.execute(add_csv_file, csv_info)

try to:

cursor.execute(query, (add_csv_file, csv_info,))

Answer 4

This is easily solved with adding the appropriate client flag in your connection as below:

import mysql.connector
from mysql.connector.constants import ClientFlag

cnx = mysql.connector.connect(user='[username]', password='[pass]', host='[host]', client_flags=[ClientFlag.LOCAL_FILES])
cursor = cnx.cursor()

This will give permission for MySQL to access the local files on your machine and then the following LOAD will work:

LoadSQL = """LOAD DATA LOCAL INFILE '%s'
    INTO TABLE %s
    FIELDS TERMINATED BY '\t'
    LINES TERMINATED BY '\n'
    IGNORE 1 LINES
    (field1, field2, field3, field4)""" % (csvfile, tabl_name)
cursor.execute(LoadSQL)
cnx.commit()
cursor.close()
cnx.close()