Passing a multiprocessing queue/dictionary/etc.. to green threads

问题

Is it safe to pass a multiprocessing object (queue, dictionary, etc...) to multiple gevent threads? Since they're not actually running concurrently, I don't think there's a problem. However, I know that gevent isn't supposed to be specifically compatible with multiprocessing.

回答1:

The benefits would likely be lost, standard threaded queue implements locks where a green thread would likely be slowed down. Thankfully, gevent often has its own but similar constructs. Check out gevent.queue

回答2:

Unfortunately, it seems like, at the moment, gevent is not compatible with objects from multiprocessing:

It is dangerous. The mp.Queue and other mp data structures utilize things like Semaphores internally: https://github.com/python/cpython/blob/master/Lib/multiprocessing/queues.py#L48

Semaphores under Linux are not fd-based and would require threaded wrappers to unblock the main loop thread. Generally speaking, if things go south, it's possible to completely block the main thread with a semaphore waiting infinitely for some event to happen.

(Quote from GitHub issues https://github.com/gevent/gevent/issues/1443)

回答3:

I would say that it is a thread safe object then it is not dangerous, but you should always think hard about it. If it isn't thread safe you need to worry about reentrancy of the methods and the consequence of the different object operations not being atomic. Some objects are stateful and they need to complete certain operations before another thread comes in.

来源：https://stackoverflow.com/questions/22162538/passing-a-multiprocessing-queue-dictionary-etc-to-green-threads