Is it possible to store the alembic connect string outside of alembic.ini?

亡梦爱人 提交于 2019-11-29 21:15:35
Tammo Heeren

I had the very same problem yesterday and found a following solution to work. I do the following in alembic/env.py:

# this is the Alembic Config object, which provides
# access to the values within the .ini file in use.
config = context.config

# this will overwrite the ini-file sqlalchemy.url path
# with the path given in the config of the main code
import config as ems_config
config.set_main_option('sqlalchemy.url', ems_config.config.get('sql', 'database'))

ems_config is an external module that holds my configuration data.

config.set_main_option(...) essentially overwrites the sqlalchemy.url key in the [alembic] section of the alembic.ini file. In my configuration I simply leave it black.

danio

Alembic documentation suggests using create_engine with the database URL (instead of modifying sqlalchemy.url in code).

Also you should modify run_migrations_offline to use the new URL. Allan Simon has an example on his blog, but in summary, modify env.py to:

  1. Provide a shared function to get the URL somehow (here it comes from the command line):

    def get_url():
        url = context.get_x_argument(as_dictionary=True).get('url')
        assert url, "Database URL must be specified on command line with -x url=<DB_URL>"
        return url
    
  2. Use the URL in offline mode:

    def run_migrations_offline():
        ...
        url = get_url()
        context.configure(
            url=url, target_metadata=target_metadata, literal_binds=True)
        ...
    
  3. Use the URL in online mode by using create_engine instead of engine_from_config:

    def run_migrations_online():
        ...
        connectable = create_engine(get_url())
        with connectable.connect() as connection:
        ...
    

So what appears to work is reimplementing engine creation in env.py, which is apparently a place for doing this kind of customizing Instead of using the sqlalchemy connect string in the ini:

engine = engine_from_config(
            config.get_section(config.config_ini_section),
            prefix='sqlalchemy.',
           poolclass=pool.NullPool)

You can replace and specify your own engine configuration:

import store
engine = store.engine

Indeed the docs seems to imply this is ok:

sqlalchemy.url - A URL to connect to the database via SQLAlchemy. This key is in fact only referenced within the env.py file that is specific to the “generic” configuration; a file that can be customized by the developer. A multiple database configuration may respond to multiple keys here, or may reference other sections of the file.

The simplest thing I could come up with to avoid commiting my user/pass was to a) add in interpolation strings to the alembic.ini file, and b) set these interpolation values in env.py

alembic.ini

sqlalchemy.url = postgresql://%(DB_USER)s:%(DB_PASS)s@35.197.196.146/nozzle-website

env.py

import os

from logging.config import fileConfig

from sqlalchemy import engine_from_config
from sqlalchemy import pool

from alembic import context

# this is the Alembic Config object, which provides
# access to the values within the .ini file in use.
config = context.config

# here we allow ourselves to pass interpolation vars to alembic.ini
# fron the host env
section = config.config_ini_section
config.set_section_option(section, "DB_USER", os.environ.get("DB_USER"))
config.set_section_option(section, "DB_PASS", os.environ.get("DB_PASS"))

...

I was looking for a while how to manage this for mutli-databases

Here is what I did. I have two databases : logs and ohlc

According to the doc, I have setup the alembic like that

alembic init --template multidb

alembic.ini

databases = logs, ohlc
[logs]
sqlalchemy.url = postgresql://botcrypto:botcrypto@localhost/logs
[ohlc]
sqlalchemy.url = postgresql://botcrypto:botcrypto@localhost/ohlc

env.py

[...]
# this is the Alembic Config object, which provides
# access to the values within the .ini file in use.
config = context.config

# Interpret the config file for Python logging.
# This line sets up loggers basically.
fileConfig(config.config_file_name)
logger = logging.getLogger('alembic.env')

# overwrite alembic.ini db urls from the config file
settings_path = os.environ.get('SETTINGS')
if settings_path:
    with open(settings_path) as fd:
        settings = conf.load(fd, context=os.environ) # loads the config.yml
    config.set_section_option("ohlc", "sqlalchemy.url", settings["databases"]["ohlc"])
    config.set_section_option("logs", "sqlalchemy.url", settings["databases"]["logs"])
else:
    logger.warning('Environment variable SETTINGS missing - use default alembic.ini configuration')
[...]

config.yml

databases:
    logs: postgresql://botcrypto:botcrypto@127.0.0.1:5432/logs
    ohlc: postgresql://botcrypto:botcrypto@127.0.0.1:5432/ohlc

usage

SETTINGS=config.yml alembic upgrade head

Hope it can helps !

Another solution is to create a template alembic.ini.dist file and to track it with your versionned code, while ignoring alembic.ini in your VCS.

Do not add any confidential information in alembic.ini.dist:

sqlalchemy.url = ...

When deploying your code to a platform, copy alembic.ini.dist to alembic.ini (this one won't be tracked by your VCS) and modify alembic.ini with the platform's credentials.

Bartoszer

As Doug T. said you can edit env.py to provide URL from somewhere else than ini file. Instead of creating new engine you can pass an additional url argument to the engine_from_config function (kwargs are later merged to options taken from ini file). In that case you could e.g. store encrypted password in ini file and decrypt it in runtime by passphrase stored in ENV variable.

connectable = engine_from_config(                 
    config.get_section(config.config_ini_section),
    prefix='sqlalchemy.',                         
    poolclass=pool.NullPool,                      
    url=some_decrypted_endpoint)                                   
易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!