“set session” in a SQLAlchemy session object

Question

I\'m using SQLAlchemy for a project, and need to be able to specify a session variable/setting for one specific call for performance reasons:

set session max         


        

Answer 1

I finally ended up doing this to get it working:

Session = sessionmaker()
...
session.connection().execute('SET max_heap_table_size = 1024 * 1024 * 64');

The SQLAlchemy event didn't seem to work, but then again, its possible I wasn't doing it exactly right.

Answer 2

Use a session event to execute an arbitrary SQL statement on each new transaction. You can also use events on the connection level, it depends on your use case.

Here is how I would do it on the session level:

Session = sessionmaker()
@event.listens_for(Session, 'before_flush')
def set_max_heap_table_size(session, transaction, connection):
    session.execute('SET max_heap_table_size = 1024 * 1024 * 64')

If you are unsure which way works for you, just try them, write some test cases and find out if that works for you.

There may be one caveat (unsure): Since the connection is not dropped but returned to the pool, the setting might persist. In this case you might also want to attach something to restore the default, e.g. on the after_flush event. I am not entirely sure on this one, you might want to experiment. If this is unnecessary, you could also use the after_begin event, but there is no real before_close event that wraps it, so it could create issues.

Answer 3

None of the above worked for me. but this worked:

engine_type: str = 'mysql+mysqlconnector'
db_user_name: str = '<some-user-name>'
db_user_password: str = '<some-user-psswd>'
db_server: str = '<db-server>'
db_name: str = '<database>'
engine: sqlalchemy.engine.Engine = create_engine(
   f'{engine_type}://{db_user_name}:{db_user_password}@{db_server}:3306/{db_name}'
)
engine.execute("SHOW VARIABLES like 'group_concat_max_len'").fetchone()
('group_concat_max_len', '1024')
engine.execute("SET SESSION group_concat_max_len=10240")
engine.execute("SHOW VARIABLES like 'group_concat_max_len'").fetchone()
('group_concat_max_len', '10240')

this must happen right before the query is performed.