A thread-safe memoize decorator

Question

I'm trying to make a memoize decorator that works with multiple threads.

I understood that I need to use the cache as a shared object between the threads, and acquire/lock the shared object. I'm of course launching the threads:

for i in range(5):
            thread = threading.Thread(target=self.worker, args=(self.call_queue,))
            thread.daemon = True
            thread.start()

where worker is:

def worker(self, call):
    func, args, kwargs = call.get()
    self.returns.put(func(*args, **kwargs))
    call.task_done()

The problem starts, of course, when I'm sending a function decorated with a memo function (like this) to many threads at the same time.

How can I implement the memo's cache as a shared object among threads?

Answer 1

The most straightforward way is to employ a single lock for the entire cache, and require that any writes to the cache grab the lock first.

In the example code you posted, at line 31, you would acquire the lock and check to see if the result is still missing, in which case you would go ahead and compute and cache the result. Something like this:

lock = threading.Lock()
...
except KeyError:
    with lock:
        if key in self.cache:
            v = self.cache[key]
        else:
            v = self.cache[key] = f(*args,**kwargs),time.time()

The example you posted stores a cache per function in a dictionary, so you'd need to store a lock per function as well.

If you were using this code in a highly contentious environment, though, it would probably be unacceptably inefficient, since threads would have to wait on each other even if they weren't calculating the same thing. You could probably improve on this by storing a lock per key in your cache. You'll need to globally lock access to the lock storage as well, though, or else there's a race condition in creating the per-key locks.