Using sets with the multiprocessing module

问题

I can't seem to share a set across processes using a Manager instance. A condensed version of my code:

from multiprocessing.managers import SyncManager
manager = SyncManager()
manager.start()
manager.register(Set)

I've also tried register(type(Set)) and register(Set()), but I'm not overly surprised that neither of them worked (the first should evaluate to Class, I think).

The exception I get in all cases is TypeError: __name__ must be set to a string object in line 675 of managers.py.

Is there a way of doing this, or do I need to investigate alternatives?

回答1:

The first argument to the SyncManager.register() class method must be a string, not a cass:

SyncManager.register('set', set)

but you'll need to register a proxy for sets as well:

from multiprocessing.managers import MakeProxyType

BaseSetProxy = MakeProxyType('BaseSetProxy', (
    '__and__', '__contains__', '__iand__', '__ior__', 
    '__isub__', '__ixor__', '__len__', '__or__', '__rand__', '__ror__', '__rsub__',
    '__rxor__', '__sub__', '__xor__', 'add', 'clear', 'copy', 'difference',
    'difference_update', 'discard', 'intersection', 'intersection_update', 'isdisjoint',
    'issubset', 'issuperset', 'pop', 'remove', 'symmetric_difference', 
    'symmetric_difference_update', 'union', 'update']
    ))
class SetProxy(BaseSetProxy):
    # in-place hooks need to return `self`, specify these manually
    def __iand__(self, value):
        self._callmethod('__iand__', (value,))
        return self
    def __ior__(self, value):
        self._callmethod('__ior__', (value,))
        return self
    def __isub__(self, value):
        self._callmethod('__isub__', (value,))
        return self
    def __ixor__(self, value):
        self._callmethod('__ixor__', (value,))
        return self

SyncManager.register('set', set, SetProxy)

来源：https://stackoverflow.com/questions/16415156/using-sets-with-the-multiprocessing-module

标签

python

asynchronous

multiprocessing