Ensuring subprocesses are dead on exiting Python program

Question

Is there a way to ensure all created subprocess are dead at exit time of a Python program? By subprocess I mean those created with subprocess.Popen().

If not, should

Answer 1

I needed a small variation of this problem (cleaning up subprocesses, but without exiting the Python program itself), and since it's not mentioned here among the other answers:

p=subprocess.Popen(your_command, preexec_fn=os.setsid)
os.killpg(os.getpgid(p.pid), 15)

setsid will run the program in a new session, thus assigning a new process group to it and its children. calling os.killpg on it thus won't bring down your own python process also.

Answer 2

This is what I did for my posix app:

When your app exists call the kill() method of this class: http://www.pixelbeat.org/libs/subProcess.py

Example use here: http://code.google.com/p/fslint/source/browse/trunk/fslint-gui#608

Answer 3

A solution for windows may be to use the win32 job api e.g. How do I automatically destroy child processes in Windows?

Here's an existing python implementation

https://gist.github.com/ubershmekel/119697afba2eaecc6330

Answer 4

The subprocess.Popen.wait() is the only way to assure that they're dead. Indeed, POSIX OS's require that you wait on your children. Many *nix's will create a "zombie" process: a dead child for which the parent didn't wait.

If the child is reasonably well-written, it terminates. Often, children read from PIPE's. Closing the input is a big hint to the child that it should close up shop and exit.

If the child has bugs and doesn't terminate, you may have to kill it. You should fix this bug.

If the child is a "serve-forever" loop, and is not designed to terminate, you should either kill it or provide some input or message which will force it to terminate.

---

Edit.

In standard OS's, you have os.kill( PID, 9 ). Kill -9 is harsh, BTW. If you can kill them with SIGABRT (6?) or SIGTERM (15) that's more polite.

In Windows OS, you don't have an os.kill that works. Look at this ActiveState Recipe for terminating a process in Windows.

We have child processes that are WSGI servers. To terminate them we do a GET on a special URL; this causes the child to clean up and exit.

Answer 5

Is there a way to ensure all created subprocess are dead at exit time of a Python program? By subprocess I mean those created with subprocess.Popen().

You could violate encapsulation and test that all Popen processes have terminated by doing

subprocess._cleanup()
print subprocess._active == []

If not, should I iterate over all of the issuing kills and then kills -9? anything cleaner?

You cannot ensure that all subprocesses are dead without going out and killing every survivor. But if you have this problem, it is probably because you have a deeper design problem.

Answer 6

help for python code: http://docs.python.org/dev/library/subprocess.html#subprocess.Popen.wait