SELECT… FOR UPDATE selecting old data after a commit

问题

Good day,

This is an update to my old post at SQL Simultaneous transactions ignore each other's locks??? DEADLOCK [InnoDB, Python], as I realized later that the issue had nothing to do with what I thought the problem was. I'm trying to create the MySQL equivilent to a T-SQL based script for a client.

I have two identical scripts, Alice and Barry running concurrently. Their goal is to

SELECT * FROM job WHERE status = 0 LIMIT 1 FOR UPDATE
UPDATE job SET status = 1 where jobIDs match, do some other stuff, and then...
COMMIT the change, and proceed with the job

The problem I have is that Alice makes the lock, reads the job, UPDATES it to status 1, but as soon as she commits the change, Barry takes up the lock and reads a status of 0, the original status... Even Alice sometimes sees status 0 just after her COMMIT

This is an small segment from my python script, but it should be enough to understand the procedure:

connection = MySQLdb.connect(host=..., user=..., [...])
cursor = connection.cursor(MySQLdb.cursors.DictCursor)
[...]
execute("START TRANSACTION")
execute("SELECT * FROM job WHERE status = %s LIMIT 1 FOR UPDATE", 0)
job_data = cursor.fetchone()
 # debug("Made lock")
if not job_data:
    connection.commit()
     # debug("No rows")
else:
     # debug("Locked row with status  "+str(job_data['status']))
    execute("SELECT status FROM job")
     # debug("Double checked status is "+str(cursor.fetchone()))
    execute("UPDATE job SET status = %s WHERE jobID = %s", 1, job_data['jobID'])
    time.sleep(5)
    execute("SELECT status FROM job")
     # debug("Status before commit "+str(cursor.fetchone()))
    connection.commit()
     # debug('Committed')
    execute("SELECT status FROM job")
     # debug("Status after commit "+str(cursor.fetchone()))

This bloated version of the original script is full of debug methods to try and understand what is happening, with time.sleep to be able to keep up with what's going on. I've commented out the debug functions in this extract to make it easier to read what is actually happening.
These are the outputs from Alice and Barry:

Alice

41,351161: Made lock
41,351161: Locked row with status  0
41,351161: Double checked status is {'status': 0}
46,352156: Status before commit {'status': 1}
46,370601: Committed
46,370601: Status after commit {'status': 1} (Sometimes Alice sees 0 here)

Barry

46,352682: Made lock
46,353184: No rows
48,365044: Made lock
48,365044: Locked row with status  0
48,365044: Double checked status is {'status': 0}
53,365062: Status before commit {'status': 1}
53,386910: Committed
53,388846: Status after commit {'status': 1}

The numbers at the start of the output are the timestamp, as SECONDS,MICROSECONDS.

Alice holds the lock for five seconds, and as soon as she COMMITS, Barry takes the lock. However, Barry sees a status of 0 (there is only one row during testing). This ends up with both thinking they can process the job.

I'm not sure why Barry reads a status of 0, after the commit. Is it rolled back? Is he reading a cache of the old value, from when he called his SELECT? Would a different isolation level help (didn't seem to)?

When I execute this code in a closed environment, without the rest of the program, it works!!! Barry reports no rows as soon as he finally gets the lock. I don't understand how it could differ? I'm guessing this means that something is wrong elsewhere in the script. But what could it be? This is a fairly isolated transaction. Calling COMMIT just before this code doesn't change anything, I thought maybe a previous transaction wasn't closed properly.
Alice and Barry are the only processes accessing the database when testing (apart from phpmyadmin).

I'm running MariaDB's InnoDB engine, with MySQLdb (mysqlclient) as the connector for Python. AUTOCOMMITis off as far as I can tell. The script is quite long, which is why I only sent a few lines of it. I'll try slicing it apart in the next few days to try and isolate the issue, or to create a small example script. I can't seem to find out how to make MariaDB print out a log of changes and locks, but if I do I'll update this post.

Any ideas, suggestions or comments would be amazing.

Have a great day!

回答1:

After following Solarflare's advice on slicing apart the program bit by bit, I eventually found the problem leading to a remote part of the code, which was functioning without error, but completely differently after the move from T-SQL to SQL.

I thought at first that the issue with the SQL server, but it was just a nasty bug. I'm now getting horrible deadlocks for some reason, yet another pleasure to look into.

Thanks for the comment and the time you spent explaining what could be wrong, I'm sorry this turned out to be just a personal silly issue

Have a great day!

来源：https://stackoverflow.com/questions/48974420/select-for-update-selecting-old-data-after-a-commit

标签

python

mysql

transactions

locking

commit