Python: GIL context - switching

Question

So, I generally have a pretty good understanding of how the Global Interpreter Lock (GIL) in Python works. Essentially, while the interpreter is running, one thread holds t

Answer 1

As Armin said, the GIL can be released inside PyEval_EvalCode. When it returns, it is of course acquired again.

The best way is just to make sure that your code can handle that. For example, incref any objects where you have C pointers to before the GIL might get released. Also, be careful if there might be cases where the Python code again calls the very same function. If you have another mutex there, you can easily end up in a dead-lock. Use recursive-safe mutexes and while waiting on them, you should release the GIL so that the original thread can releases such mutexes.

Answer 2

Yes, the interpreter can always release the GIL; it will give it to some other thread after it has interpreted enough instructions, or automatically if it does some I/O. Note that since recent Python 3.x, the criteria is no longer based on the number of executed instructions, but on whether enough time has elapsed.

To get a different effect, you'd need a way to acquire the GIL in "atomic" mode, by asking the GIL not to be released until you release it explicitly. This is impossible so far (but see https://bitbucket.org/arigo/cpython-withatomic for an experimental version).