Adding indexes to SQLAlchemy models after table creation

Question

I have a flask-sqlalchemy model:

class MyModel(db.Model):
__tablename__ = \'targets\'
id = db.Column(db.Integer, primary_key=True)
url = db.Column(db.String(2048         


        

Answer 1

I am not sure if this conforms to best practices but I found Alembic would notify me of Indexes in the __table_args__ but not actually make them for me during migrations. I made this small script that can generate new indexes found in the __table_args__ property. It makes use of Index.create() as mentioned above, but will generate new indexes if they do not exist.

def create_indexes(db, drop_index=False):
    """
    Creates all indexes on models in project if they do not exists already. Assumes all models
    inherit from RequiredFields class, otherwise will need to adjust search for subclasses. If the index
    exists SQLAlchemy throws an error and we assume everything is ok. 
    :param db: The app db object, acts as the engine param for the Index.create()
    :param drop_index: specifies whether the indexes should be dropped or created
    :return:
    """
    from application.base_models import RequiredFields
    from sqlalchemy import Index
    from sqlalchemy.exc import ProgrammingError
    for klass in RequiredFields.__subclasses__():
        if hasattr(klass, '__table_args__'):
            for item in getattr(klass, '__table_args__'):
                if isinstance(item, Index):
                    try:
                        if not drop_index:
                            item.create(db.engine)
                        else:
                            item.drop(db.engine)
                    except ProgrammingError:  # If index exists, creation fails on error
                        pass
    return

Here is a sample class showing the indexes.

class MyModel(RequiredFields):

    __table_args__ = (
         db.Index( ... ),
         db.Index( ... ),
    )

Answer 2

Please note that this is incorrect and over-complicated answer

The right way is to use index.create as it was said here.

---

First of all make sure that you have latest snapshot of your database and is able to restore database from this snapshot.

For medium and large size projects (the ones that you might need to support several versions at the same time and are installed on multiple environments) there is special procedure which is part of database management lifecycle called "database migration". DB migrations includes changes to existing schema. SQLAlchemy doesn't support migration out of the box.

There are two SQLAlchemy compatible database migration tools available:

• Alembic
• SQLAlchemy-Migrate

See more information and links to these tools in SQLAlchemy documentation page: Altering Schemas through Migrations.

But if your are working on small project I would suggest to manually run ALTER TABLE DDL query from the database command line utility or through connection.execute() in python script.

In the production application I'm working at right now, we support only one latest version of application. For every database schema change we do the following steps:

• make a snapshot of the production database
• load this snapshot on development environment
• update sqlalchemy data model module
• prepare and run alter table query and save this query for later
• make other related changes to the code
• run tests on dev environment
• deploy latest version of the code to production
• do alter table on production

Also I'm using the following trick for generating create table/index queries: I point my application to brand new database, enable logging of sqlalchemy queries and run metadata.create_all() - so in logs (or STDOUT) I see create query generated by sqlalchemy

Depending on the database system you are using index creation query will be little different. Generic query would look like this:

create index targets_i on targets(url);

Answer 3

Since the question was asked, support has been added for this.

Now you can just add index=True to an existing column, and auto-generate the migration.

Checked on the following package versions:

alembic==1.0.10
SQLAlchemy==1.3.4
SQLAlchemy-Utils==0.34.0
Flask-SQLAlchemy==2.4.0
Flask-Migrate==2.5.2

Answer 4

Call create() on the Index:

index.create()

http://docs.sqlalchemy.org/en/latest/core/constraints.html#sqlalchemy.schema.Index.create

Answer 5

Given the model class from the original question.

class MyModel(db.Model):
    __tablename__ = 'targets'
    id = db.Column(db.Integer, primary_key=True)
    url = db.Column(db.String(2048))

You cannot just add index=True because even if you called db.Model.metadata.create_all() the index will not be created on an already created table.

Instead, you need to create an independent Index object, and then create it. It will look something like this:

class MyModel(db.Model):
    __tablename__ = 'targets'
    id = db.Column(db.Integer, primary_key=True)
    url = db.Column(db.String(2048))

mymodel_url_index = Index('mymodel_url_idx', MyModel.url)

if __name__ == '__main__':
    mymodel_url_index.create(bind=engine)

Now where engine comes from will be up to your sqlalchemy configuration, but this code should convey the gist of what needs to happen.

Answer 6

Use flask-migrate.It's cool. After you add the index,just use this command:

python manage.py db migrate

Everything works fine