PyGILState_Ensure() Causing Deadlock

Question

I\'m writing a Python extension in C++, wrapping a third-party library I do not control. That library creates a thread Python knows nothing about, and from that thread, calls a

Answer 1

I have wrapped C++ observers in Python. If you are using boost then you can call PyEval_InitThreads() in BOOST_PYTHON_MODULE:

BOOST_PYTHON_MODULE(eapipy)
{
     boost::shared_ptr gil(new Python::InitialisePythonGIL());
....
}

Then I use a class to control calling back into Python from C++.

struct PyLockGIL
{

    PyLockGIL()
        : gstate(PyGILState_Ensure())
    { 
    }

    ~PyLockGIL()
    {
        PyGILState_Release(gstate);
    }

    PyLockGIL(const PyLockGIL&) = delete;
    PyLockGIL& operator=(const PyLockGIL&) = delete;

    PyGILState_STATE gstate;
};

If you are calling into C++ for any length of time you can also relinquish the GIL:

struct PyRelinquishGIL
{
    PyRelinquishGIL()
        : _thread_state(PyEval_SaveThread())
    {
    }
    ~PyRelinquishGIL()
    {
        PyEval_RestoreThread(_thread_state);
    }

    PyRelinquishGIL(const PyLockGIL&) = delete;
    PyRelinquishGIL& operator=(const PyLockGIL&) = delete;

    PyThreadState* _thread_state;
};

Our code is multi-threaded and this approach works well.