dump csv from sqlalchemy

Question

For some reason, I want to dump a table from a database (sqlite3) in the form of a csv file. I\'m using a python script with elixir (based on sqlalchemy) to modify the datab

Answer 1

Modifying Peter Hansen's answer here a bit, to use SQLAlchemy instead of raw db access

import csv
outfile = open('mydump.csv', 'wb')
outcsv = csv.writer(outfile)
records = session.query(MyModel).all()
[outcsv.writerow([getattr(curr, column.name) for column in MyTable.__mapper__.columns]) for curr in records]
# or maybe use outcsv.writerows(records)

outfile.close()

Answer 2

There are numerous ways to achieve this, including a simple os.system() call to the sqlite3 utility if you have that installed, but here's roughly what I'd do from Python:

import sqlite3
import csv

con = sqlite3.connect('mydatabase.db')
outfile = open('mydump.csv', 'wb')
outcsv = csv.writer(outfile)

cursor = con.execute('select * from mytable')

# dump column titles (optional)
outcsv.writerow(x[0] for x in cursor.description)
# dump rows
outcsv.writerows(cursor.fetchall())

outfile.close()

Answer 3

with open('dump.csv', 'wb') as f:
    out = csv.writer(f)
    out.writerow(['id', 'description'])

    for item in session.query(Queue).all():
        out.writerow([item.id, item.description])

I found this to be useful if you don't mind hand-crafting your column labels.

Answer 4

import csv

f = open('ratings.csv', 'w')
out = csv.writer(f)
out.writerow(['id', 'user_id', 'movie_id', 'rating'])

for item in db.query.all():
    out.writerow([item.username, item.username, item.movie_name, item.rating])
f.close()

Answer 5

I spent a lot of time searching for a solution to this problem and finally created something like this:

from sqlalchemy import inspect

with open(file_to_write, 'w') as file:
    out_csv = csv.writer(file, lineterminator='\n')

    columns = [column.name for column in inspect(Movies).columns][1:]
    out_csv.writerow(columns)

    session_3 = session_maker()

    extract_query = [getattr(Movies, col) for col in columns]
    for mov in session_3.query(*extract_query):
        out_csv.writerow(mov)

    session_3.close()

It creates a CSV file with column names and a dump of the entire "movies" table without "id" primary column.

Answer 6

I adapted the above examples to my sqlalchemy based code like this:

import csv
import sqlalchemy as sqAl

metadata = sqAl.MetaData()
engine = sqAl.create_engine('sqlite:///%s' % 'data.db')
metadata.bind = engine

mytable = sqAl.Table('sometable', metadata, autoload=True)
db_connection = engine.connect()

select = sqAl.sql.select([mytable])
result = db_connection.execute(select)

fh = open('data.csv', 'wb')
outcsv = csv.writer(fh)

outcsv.writerow(result.keys())
outcsv.writerows(result)

fh.close

This works for me with sqlalchemy 0.7.9. I suppose that this would work with all sqlalchemy table and result objects.