Django Celery cache lock did not work?

Question

I am trying to use Django cache to implement a lock mechanism. In Celery offical site, it claimed Django cache work fine for this. However, in my experence, it did not work.

Answer 1

The problem is that Django makes no guarantees as to the atomicity of .add(). Whether or not .add() will in fact be atomic depends on the backend you are using. With a FileBasedCache, .add() is not atomic:

def add(self, key, value, timeout=DEFAULT_TIMEOUT, version=None):
    if self.has_key(key, version):
        return False
    self.set(key, value, timeout, version)
    return True

Worker A executing .add() could be preempted after self.has_key(...) but before self.set(...). Worker B executing .add() in one shot would successfully set the key and return True. When worker A resumes, it would also set the key and return True.

This issue report indicates that the example code you looked at assumes that the backend is Memcached. If you use Memcached or a backend that supports an atomic .add() then it should work.