Python: Update Local Variable in a Parallel Process from Parent Program

Question

I am relatively new to programming, and what I am asking might be a task that is not possible. What I want to do is start a parallel process, which will continuously run until

Answer 1

Nothing wrong with a queue here, but it's probably more idiomatic to use "shared memory" instead. Less overhead. Here's an example self-contained program:

import time

def loop(i):
    while 1:
        print i.value
        i.value += 1
        time.sleep(1)

if __name__ == "__main__":
    from multiprocessing import Process, Value

    i = Value("i", 1)  # "i" for integer, initial value 1
    p = Process(target=loop, args=(i,))
    p.start()
    for base in range(100, 600, 100):
        time.sleep(2)
        i.value = base

That will probably ;-) display:

1
2
100
101
200
201
300
301
400
401
500
501
502
503
504
505
506
...

But caution: in return for being speedier, this is also more brittle. The kinds of data you can share this way are basically inherited from the C language, and aren't generally as rich as Python data types. In the example above, the type code "i" means we're sharing a signed C int, which is usually a 32-bit signed integer. If, for example, i.value reaches 2147483647, and we add 1 to it, it will suddenly become -2147483648 (yup, negative!). Python ints are unbounded, but C ints aren't.

Answer 2

You can use a queue to pass data between the processes:

from multiprocessing import Process, Queue
from loop import loop

i = 1
q = Queue()
q.put(i)
p = Process(target=loop, args=(q, ))
p.start()

Whenever you want to transmit a new value of i to the other process, just put it in the queue.

Change your loop.py module accordingly:

def loop(q):
  while True:
    i = q.get()
    print i

In your main code, you can put new values for the process:

while True:
  i+=1
  q.put(i)
  time.sleep(1)

Answer 3

Use a queue. (python 2 is Queue)

There are a few ways to consume from the queue. The naive implementation (i.e. the one that has a race condition) is simply to do if q.empty(). This is bad practice but wouldn't hurt you since it's not mission critical if you miss a tick.

The better method is to use exceptions for flow control:

q = queue.Queue()
try:
    q.get(False) #non-blocking get()
    #update value of i, etc
except queue.Empty:
    #do stuff here as before