How to lock a critical section in Django?

Question

I can\'t find a good clean way to lock a critical section in Django. I could use a lock or semaphore but the python implementation is for threads only, so if the production

Answer 1

You could use simple file locking as a mutual exclusion mechanism, see my recipe here. It won't suit all scenarios, but then you haven't said much about why you want to use this type of locking.

Answer 2

I did not write this article, but I found it supremely helpful faced with this same problem:

http://chris-lamb.co.uk/2010/06/07/distributing-locking-python-and-redis/

Basically you use a Redis server to create a central server that exposes the locking functionality.

Answer 3

Use the Django builtin select_for_update function.
https://docs.djangoproject.com/en/1.8/ref/models/querysets/#select-for-update
From the docs:
Returns a queryset that will lock rows until the end of the transaction, generating a SELECT ... FOR UPDATE SQL statement on supported databases.

For example:

entries = Entry.objects.select_for_update().filter(author=request.user)

All matched entries will be locked until the end of the transaction block, meaning that other transactions will be prevented from changing or acquiring locks on them.

Answer 4

I ended up going with a solution I made myself involving file locking. If anyone here ends up using it remember that advisory locks and NFS don't mix well, so keep it local. Also, this is a blocking lock, if you want to mess around with loops and constantly checking back then there is instructions in the code.

import os
import fcntl

class DjangoLock:

    def __init__(self, filename):
        self.filename = filename
        # This will create it if it does not exist already
        self.handle = open(filename, 'w')

    # flock() is a blocking call unless it is bitwise ORed with LOCK_NB to avoid blocking 
    # on lock acquisition.  This blocking is what I use to provide atomicity across forked
    # Django processes since native python locks and semaphores only work at the thread level
    def acquire(self):
        fcntl.flock(self.handle, fcntl.LOCK_EX)

    def release(self):
        fcntl.flock(self.handle, fcntl.LOCK_UN)

    def __del__(self):
        self.handle.close()

Usage:

lock = DJangoLock('/tmp/djangolock.tmp')
lock.acquire()
try:
    pass
finally:
    lock.release()

Answer 5

If you use RDBMS, you can use its "LOCK" mechanism. For example, while one "SELECT FOR UPDATE" transaction locks a row, the other "SELECT FOR UPDATE" transactions with the row must wait.

# You can use any Python DB API.
[SQL] BEGIN;

[SQL] SELECT col_name FROM table_name where id = 1 FOR UPDATE;

[Process some python code]

[SQL] COMMIT;

Answer 6

You need a distributed lock manager at the point where your app suddenly needs to run on more than one service. I wrote elock for this purpose. There are bigger ones and others have chosen to ignore every suggestion and done the same with memcached.

Please don't use memcached for anything more than light advisory locking. It is designed to forget stuff.

I like to pretend like filesystems don't exist when I'm making web apps. Makes scale better.