I know that executemany
can be used to conveniently add new entries to a database; useful to reduce the Python method-call overheads compared to single execut
what you have is perfect except that you should use zip(conts,ids) where conts and ids are lists. This does the rearrangement automatically for you.
You need to pass a sequence of sequences ([[cont,id], [cont,id], [cont,id], ...]
, not [cont, cont, cont, ...], [id, id, id, ..]
):
for path in paths:
params = []
for data in some_computation(path):
if len(data_ids) >= CHUNKSIZE:
c.executemany("UPDATE TABLENAME SET cont=? WHERE id=?", params)
cnx.commit()
params = []
params.append([data[1], data[0]])
if params:
c.executemany("UPDATE TABLENAME SET cont=? WHERE id=?", params)
cnx.commit()