fcntl.flock - how to implement a timeout?

Question

I am using python 2.7

I want to create a wrapper function around fcntl.flock() that will timeout after a set interval:

wrapper_function(timeout):
         


        

Answer 1

For Python 3.5+, Glenn Maynard's solution no longer works because of PEP-475. This is a modified version:

import signal, errno
from contextlib import contextmanager
import fcntl

@contextmanager
def timeout(seconds):
    def timeout_handler(signum, frame):
        # Now that flock retries automatically when interrupted, we need
        # an exception to stop it
        # This exception will propagate on the main thread, make sure you're calling flock there
        raise InterruptedError

    original_handler = signal.signal(signal.SIGALRM, timeout_handler)

    try:
        signal.alarm(seconds)
        yield
    finally:
        signal.alarm(0)
        signal.signal(signal.SIGALRM, original_handler)

with timeout(1):
    f = open("test.lck", "w")
    try:
        fcntl.flock(f.fileno(), fcntl.LOCK_EX)
    except InterruptedError:
        # Catch the exception raised by the handler
        # If we weren't raising an exception, flock would automatically retry on signals
        print("Lock timed out")

Answer 2

Timeouts for system calls are done with signals. Most blocking system calls return with EINTR when a signal happens, so you can use alarm to implement timeouts.

Here's a context manager that works with most system calls, causing IOError to be raised from a blocking system call if it takes too long.

import signal, errno
from contextlib import contextmanager
import fcntl

@contextmanager
def timeout(seconds):
    def timeout_handler(signum, frame):
        pass

    original_handler = signal.signal(signal.SIGALRM, timeout_handler)

    try:
        signal.alarm(seconds)
        yield
    finally:
        signal.alarm(0)
        signal.signal(signal.SIGALRM, original_handler)

with timeout(1):
    f = open("test.lck", "w")
    try:
        fcntl.flock(f.fileno(), fcntl.LOCK_EX)
    except IOError, e:
        if e.errno != errno.EINTR:
            raise e
        print "Lock timed out"

Answer 3

I'm sure there are several ways, but how about using a non-blocking lock? After some n attempts, give up and exit?

To use non-blocking lock, include the fcntl.LOCK_NB flag, as in:

fcntl.flock(self.__lock_file.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB)

Answer 4

I'm a fan of shelling out to flock here, since attempting to do a blocking lock with a timeout requires changes to global state, which makes it harder to reason about your program, especially if threading is involved.

You could fork off a subprocess and implement the alarm as above, or you could just exec http://man7.org/linux/man-pages/man1/flock.1.html

import subprocess
def flock_with_timeout(fd, timeout, shared=True):
    rc = subprocess.call(['flock', '--shared' if shared else '--exclusive', '--timeout', str(timeout), str(fd)])
    if rc != 0:
        raise Exception('Failed to take lock')

If you have a new enough version of flock you can use -E to specify a different exit code for the command otherwise succeeding, but failed to take the lock after a timeout, so you can know whether the command failed for some other reason instead.