executemany

Python + MySQLdb executemany

由 别来无恙 提交于 2021-02-19 01:26:44
问题 I'm using Python and its MySQLdb module to import some measurement data into a Mysql database. The amount of data that we have is quite high (currently about ~250 MB of csv files and plenty of more to come). Currently I use cursor.execute(...) to import some metadata. This isn't problematic as there are only a few entries for these. The problem is that when I try to use cursor.executemany() to import larger quantities of the actual measurement data, MySQLdb raises a TypeError: not all

pymssql executemany insert value very slow

由 我怕爱的太早我们不能终老 提交于 2021-01-28 18:16:14
问题 python-2.7.15, pymssql-2.1.4, SQL_Server-2018, Windows 10 Pro, MS-Office-2016 import time import csv import pymssql db_settings = { "host" : "127.0.0.1", "port" : "1433", "user" : "sa", "password" : "********", "database" : "testdb", "charset" : "utf8" } conn = pymssql.connect(**db_settings) cursor = conn.cursor() ff = csv.reader(open('base.csv', 'r')) sql = """ BEGIN INSERT INTO Base([name], [year], [update], [status], [timeline], [language], [pic]) VALUES (%s, %s, %s, %s, %s, %s, %s) END ""

Values are not inserted into MySQL table using pool.apply_async in python2.7

由 假如想象 提交于 2020-01-06 06:51:16
问题 I am trying to run the following code to populate a table in parallel for a certain application. First the following function is defined which is supposed to connect to my db and execute the sql command with the values given (to insert into table). def dbWriter(sql, rows) : # load cnf file MYSQL_CNF = os.path.abspath('.') + '/mysql.cnf' conn = MySQLdb.connect(db='dedupe', charset='utf8', read_default_file = MYSQL_CNF) cursor = conn.cursor() cursor.executemany(sql, rows) conn.commit() cursor

Executemany confusion

由 不羁岁月 提交于 2019-12-12 01:33:55
问题 Ok, so I have a function that selects certain rows in a sqlite database based on input from a plugin. I got the plugin to select and fetch rows when just one statement is involved, but since I want to add some flexibility to this, I tried making the function use executemany when encountering lists or tuples. Yet, despite all the things I have fiddled and changed, I a still unable to get this to work, either because the sqlite statement is treating each character in the string as a binding, or

psycopg2 executemany with simple list?

由 ε祈祈猫儿з 提交于 2019-12-10 22:07:06
问题 I'm trying to use psycopg2 executemany for a simple multi-insert but I can only make it work using dict and not "plain" sequence of values: # given: values = [1, 2, 3] ; cursor = conn.cursor() # this raises TypeError: 'int' object does not support indexing: cursor.executemany('INSERT INTO t (col_a) VALUES ( %s )', values) # I also tried encapsulating my 'values' into a tuple/list but it gives another exception (TypeError: not all arguments converted during string formatting). # while this is

executemany for MySQLdb error for large number of rows

由 ∥☆過路亽．° 提交于 2019-12-10 10:14:40
问题 I'm currently running a script to insert values (a list of tuples) into a MySQL database, using the execute many function. When I use a small number of rows (`1000), the script runs fine. When I use around 40,000 rows, I receive the following errors: cursor.executemany( stmt, trans_frame) Traceback (most recent call last): File "C:\Python27\lib\site-packages\IPython\core\interactiveshell.py", line 2538, in run_code exec code_obj in self.user_global_ns, self.user_ns File "<ipython-input-1

Including DB function calls in python MySQLdb executemany()

由 孤者浪人 提交于 2019-12-08 01:16:21
问题 When I try to run statements like: cursor.executemany("""INSERT INTO `test` (`id`,`data`,`time_added`) VALUES (%s, %s, NOW())""", [(i.id, i.data) for i in items]) MySQLdb seems to choke on the ) in NOW(), when it expands the list of rows to be inserted because it sees that parenthesis as the end of the value block. That is, the queries look like: ('1', 'a', NOW(), ('2','b', NOW(), ('3','c',NOW()) And MYSQL reports a syntax error. Instead, they should look like: ('1', 'a', NOW()), ('2','b',

Including DB function calls in python MySQLdb executemany()

由 偶尔善良 提交于 2019-12-06 12:45:20
When I try to run statements like: cursor.executemany("""INSERT INTO `test` (`id`,`data`,`time_added`) VALUES (%s, %s, NOW())""", [(i.id, i.data) for i in items]) MySQLdb seems to choke on the ) in NOW(), when it expands the list of rows to be inserted because it sees that parenthesis as the end of the value block. That is, the queries look like: ('1', 'a', NOW(), ('2','b', NOW(), ('3','c',NOW()) And MYSQL reports a syntax error. Instead, they should look like: ('1', 'a', NOW()), ('2','b', NOW()), ('3','c',NOW()) There should be some way to escape the NOW(), but I can't figure out how. Adding

executemany for MySQLdb error for large number of rows

由 ぃ、小莉子 提交于 2019-12-06 00:42:05
I'm currently running a script to insert values (a list of tuples) into a MySQL database, using the execute many function. When I use a small number of rows (`1000), the script runs fine. When I use around 40,000 rows, I receive the following errors: cursor.executemany( stmt, trans_frame) Traceback (most recent call last): File "C:\Python27\lib\site-packages\IPython\core\interactiveshell.py", line 2538, in run_code exec code_obj in self.user_global_ns, self.user_ns File "<ipython-input-1-66b44e71cf5a>", line 1, in <module> cursor.executemany( stmt, trans_frame) File "C:\Python27\lib\site
订阅 executemany