Managing connection to redis from Python

Question

I\'m using redis-py in my python application to store simple variables or lists of variables in a Redis database, so I thought it would be better to create a co

Answer 1

@sg1990 what if you have 10.000 users requiring redis at the same time? They cannot share a single connection and you've just created yourself a bottleneck.

With a pool of connections you can create an arbitrary number of connections and simply use get_connection() and release(), from redis-py docs.

A connection per user is a huge overkill, since every connection needs to maintain an open socket. This way you'd automatically decrease a number of e.g. concurrent websocket users that your machine can handle by half.

Answer 2

Python uses a reference counter mechanism to deal with objects, so at the end of the blocks, the my_server object will be automatically destroyed and the connection closed. You do not need to close it explicitly.

Now this is not how you are supposed to manage Redis connections. Connecting/disconnecting for each operation is too expensive, so it is much better to maintain the connection opened. With redis-py it can be done by declaring a pool of connections:

import redis

POOL = redis.ConnectionPool(host='10.0.0.1', port=6379, db=0)

def getVariable(variable_name):
    my_server = redis.Redis(connection_pool=POOL)
    response = my_server.get(variable_name)
    return response

def setVariable(variable_name, variable_value):
    my_server = redis.Redis(connection_pool=POOL)
    my_server.set(variable_name, variable_value)

Please note connection pool management is mostly automatic and done within redis-py.