Monitor concurrency (sharing object across processes) in Python

怎甘沉沦 提交于 2020-01-04 07:26:08

问题


I'm new here and I'm Italian (forgive me if my English is not so good). I am a computer science student and I am working on a concurrent program project in Python. We should use monitors, a class with its methods and data (such as condition variables). An instance (object) of this class monitor should be shared accross all processes we have (created by os.fork o by multiprocessing module) but we don't know how to do. It is simpler with threads because they already share memory but we MUST use processes. Is there any way to make this object (monitor) shareable accross all processes? Hoping I'm not saying nonsenses...thanks a lot to everyone for tour attention. Waiting answers. Lorenzo


回答1:


As far as "sharing" the instance, I believe the instructor wants you to make your monitor's interface to its local process such that it's as if it were shared (a la CORBA).

Look into the absolutely fantastic documentation on multiprocessing's Queue:

from multiprocessing import Process, Queue

def f(q):
    q.put([42, None, 'hello'])

if __name__ == '__main__':
    q = Queue()
    p = Process(target=f, args=(q,))
    p.start()
    print q.get()    # prints "[42, None, 'hello']"
    p.join()

You should be able to imagine how your monitor's attributes might be propagated among the peer processes when changes are made.




回答2:


shared memory between processes is usually a poor idea; when calling os.fork(), the operating system marks all of the memory used by parent and inherited by the child as copy on write; if either process attempts to modify the page, it is instead copied to a new location that is not shared between the two processes.

This means that your usual threading primitives (locks, condition variables, et-cetera) are not useable for communicating across process boundaries.

There are two ways to resolve this; The preferred way is to use a pipe, and serialize communication on both ends. Brian Cain's answer, using multiprocessing.Queue, works in this exact way. Because pipes do not have any shared state, and use a robust ipc mechanism provided by the kernel, it's unlikely that you will end up with processes in an inconsistent state.

The other option is to allocate some memory in a special way so that the os will allow you to use shared memory. the most natural way to do that is with mmap. cPython won't use shared memory for native python object's though, so you would still need to sort out how you will use this shared region. A reasonable library for this is numpy, which can map the untyped binary memory region into useful arrays of some sort. Shared memory is much harder to work with in terms of managing concurrency, though; since there's no simple way for one process to know how another processes is accessing the shared region. The only time this approach makes much sense is when a small number of processes need to share a large volume of data, since shared memory can avoid copying the data through pipes.



来源:https://stackoverflow.com/questions/11809438/monitor-concurrency-sharing-object-across-processes-in-python

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!