how to kill zombie processes created by multiprocessing module?

Question

I\'m very new to multiprocessing module. And I just tried to create the following: I have one process that\'s job is to get message from RabbitMQ and pass it to

Answer 1

Using multiprocessing.active_children is better than Process.join. The function active_children cleans any zombies created since the last call to active_children. The method join awaits the selected process. During that time, other processes can terminate and become zombies, but the parent process will not notice, until the awaited method is joined. To see this in action:

import multiprocessing as mp
import time


def main():
    n = 3
    c = list()
    for i in xrange(n):
        d = dict(i=i)
        p = mp.Process(target=count, kwargs=d)
        p.start()
        c.append(p)
    for p in reversed(c):
        p.join()
        print('joined')


def count(i):
    print('{i} going to sleep'.format(i=i))
    time.sleep(i * 10)
    print('{i} woke up'.format(i=i))


if __name__ == '__main__':
    main()

The above will create 3 processes that terminate 10 second apart each. As the code is, the last process is joined first, so the other two, which terminated earlier, will be zombies for 20 seconds. You can see them with:

ps aux | grep Z

There will be no zombies if the processes are awaited in the sequence that they will terminate. Remove the reversed to see this case. However, in real applications we rarely know the sequence that children will terminate, so using join will result in some zombies.

The alternative active_children does not leave any zombies. In the above example, replace the loop for p in reversed(c): with:

while True:
    time.sleep(1)
    if not mp.active_children():
        break

and see what happens.

Answer 2

A couple of things:

1. Make sure the parent joins its children, to avoid zombies. See Python Multiprocessing Kill Processes

2. You can check whether a child is still running with the is_alive() member function. See http://docs.python.org/2/library/multiprocessing.html#multiprocessing.Process

Answer 3

Use active_children. multiprocessing.active_children