performance of insert with python and sqlite3

Question

I\'m doing big batch inserts into an SQLite3 database and I\'m trying to get a sense for what sort of performance I should be expecting versus what I\'m actually seeing.

Answer 1

As I understand the main reason of bad performance is time you waste to commit many SQLite transactions. What to do?

Drop the indexes, then

PRAGMA synchronous = OFF (or NORMAL)

Insert blocks of N rows (define N, try N=5000 to start). Before inserting block do

BEGIN TRANSACTION

after inserting do

COMMIT

See also http://www.sqlite.org/faq.html#q19