SQLAlchemy and Multiple Databases

Question

I have an assortment of similar (but not identical) databases, and would like to use SQLAlchemy as a way to \"standardize\" access. The databases can differ very slightly, such

Answer 1

As per the requests to my initial question, I've taken my third edit and made it my answer. As I'm not sure of the proper protocols, I've left the third edit in place above. If you've already read EDIT3, then you've read what I have as an answer.

I've managed to spend a little more time on this. I've set up the project in the following way:

Databases
    __init__.py
    databases.py
    settings.py
    DB1
        __init__.py
        models.py
    ...
    DB3
        __init__.py
        models.py

At present, I have a tuple of databases that are 'installed' in the settings.py file. Each DB entry is specified in INSTALLED_DATABASES = ('DB1', ..., 'DB3'). As I complete more models, they get added to the list of tuples. This allows me to add or remove content as I go.

I have the engine and sessions setup inside the models.py file, and have the __init.py__ file for each database set to from models import *.

In the databases.py file I have the following

class Databases(object):
    def __init__(self):
        for database in INSTALLED_DATABASES:
            setattr(self, database, __import__(database))

I can now use these via:

from databases import Databases

db = Databases()

for qr in db.DB1.query(db.DB1.User):
    print qr.userid, qr.username

SQLAlchemy is allowing me to manually specify column names when defining the models, which is a huge bonus to the standardization I wanted.

I've got a lot of work ahead of me still. I would like to create objects that force model validation (ie, is a field present? does a non-present field have a default value? etc.) and better incorporate how this works with my IDE (it kind of doesn't at present). But I'm off to the right track. I figured I'd update this for anyone who might by chance be wondering how to do the same thing I was.

Sorry this has become so long!

Cheers!

Answer 2

Your solution looks pretty good. Here's what I did.

I have a package named connectors, and in it a module for each db as well as a settings file.

Each of these connector modules creates its connection string and its engine, along with the declarative base and classes for the tables.

Then there is a method loadSession that returns the session (this one I got from a tutorial or another post here somewhere, cant recall exactly) and another one I added that returns the engine in case I want to do something with that.

So then in some other module of the program, I would do something like this

from connectors import x, y, z

x_ses = x.loadSession()
y_ses = y.loadSession()
z_ses = z.loadSession()

xq = x_ses.query(...)
yq = y_ses.query(...)