Python and Django OperationalError (2006, 'MySQL server has gone away')

Question

Original: I have recently started getting MySQL OperationalErrors from some of my old code and cannot seem to trace back the problem. Since it was working before, I thought

Answer 1

Sometimes if you see "OperationalError: (2006, 'MySQL server has gone away')", it is because you are issuing a query that is too large. This can happen, for instance, if you're storing your sessions in MySQL, and you're trying to put something really big in the session. To fix the problem, you need to increase the value of the max_allowed_packet setting in MySQL.

The default value is 1048576.

So see the current value for the default, run the following SQL:

select @@max_allowed_packet;

To temporarily set a new value, run the following SQL:

set global max_allowed_packet=10485760;

To fix the problem more permanently, create a /etc/my.cnf file with at least the following:

[mysqld]
max_allowed_packet = 16M

After editing /etc/my.cnf, you'll need to restart MySQL or restart your machine if you don't know how.

Answer 2

In my opinion, the must common issue regarding such warning, is the fact that your application has reached the wait_timeout value of MySQL.

I had the same problem with a PythonFLASK app which I developed and I solved very easy.

P.S: I was using MySQL 5.7.14

$ grep timeout /etc/mysql/mysql.conf.d/mysqld.cnf 
# https://support.rackspace.com/how-to/how-to-change-the-mysql-timeout-on-a-server/
# wait = timeout for application session (tdm)
# inteactive = timeout for keyboard session (terminal)
# 7 days = 604800s / 4 hours = 14400s 
wait_timeout = 604800
interactive_timeout = 14400

One important observation: if you search for the variables via MySQL batch mode, the values will appear as it is. But If you perform "SHOW VARIABLES LIKE 'wait%';" or "SHOW VARIABLES LIKE 'interactive%';", the value configured for 'interactive_timeout', will appear to both variables, and I don't know why, but the fact is, that the values configured for each variable at '/etc/mysql/mysql.conf.d/mysqld.cnf', will be respected by MySQL process.

Regards!

Answer 3

I had this problem and did not have the option to change my configuration. I finally figured out that the problem was occurring 49500 records in to my 50000-record loop, because that was the about the time I was trying again (after having tried a long time ago) to hit my second database.

So I changed my code so that every few thousand records, I touched the second database again (with a count() of a very small table), and that fixed it. No doubt "ping" or some other means of touching the database would work, as well.

Answer 4

Firstly, You should make sure the MySQL session & global enviroments wait_timeout and interactive_timeout values. And secondly Your client should try to reconnect to the server below those enviroments values.

Answer 5

For me this was happening in debug mode.

So I tried Persistent connections in debug mode, checkout the link: Django - Documentation - Databases - Persistent connections.

In settings:

'default': {
       'ENGINE': 'django.db.backends.mysql',
       'NAME': 'dbname',
       'USER': 'root',
       'PASSWORD': 'root',
       'HOST': 'localhost',
       'PORT': '3306',
       'CONN_MAX_AGE': None
    },

Answer 6

I've been struggling with this issue too. I don't like the idea of increasing timeout on mysqlserver. Autoreconnect with CONNECTION_MAX_AGE doesn't work either as it was mentioned. Unfortunately I ended up with wrapping every method that queries the database like this

def do_db( callback, *arg, **args):
    try:
        return callback(*arg, **args)
    except (OperationalError, InterfaceError) as e:  # Connection has gone away, fiter it with message or error code if you could catch another errors
        connection.close()
        return callback(*arg, **args)

do_db(User.objects.get, id=123)  # instead of User.objects.get(id=123)

As you can see I rather prefer catching the exception than pinging the database every time before querying it. Because catching an exception is a rare case. I would expect django to reconnect automatically but they seemed to refused that issue.