Python: How to determine subprocess children have all finished running

问题

I am trying to detect when an installation program finishes executing from within a Python script. Specifically, the application is the Oracle 10gR2 Database. Currently I am using the subprocess module with Popen. Ideally, I would simply use the wait() method to wait for the installation to finish executing, however, the documented command actually spawns child processes to handle the actual installation. Here is some sample code of the failing code:

import subprocess
OUI_DATABASE_10GR2_SUBPROCESS = ['sudo',
                                 '-u',
                                 'oracle',
                                 os.path.join(DATABASE_10GR2_TMP_PATH,
                                              'database',
                                              'runInstaller'),
                                 '-ignoreSysPrereqs',
                                 '-silent',
                                 '-noconfig',
                                 '-responseFile '+ORACLE_DATABASE_10GR2_SILENT_RESPONSE]
oracle_subprocess = subprocess.Popen(OUI_DATABASE_10GR2_SUBPROCESS)
oracle_subprocess.wait()

There is a similar question here: Killing a subprocess including its children from python, but the selected answer does not address the children issue, instead it instructs the user to call directly the application to wait for. I am looking for a specific solution that will wait for all children of the subprocess. What if there are an unknown number of subprocesses? I will select the answer that addresses the issue of waiting for all children subprocesses to finish.

More clarity on failure: The child processes continue executing after the wait() command since that command only waits for the top level process (in this case it is 'sudo'). Here is a simple diagram of the known child processes in this problem: Python subprocess module -> Sudo -> runInstaller -> java -> (unknown)

回答1:

Ok, here is a trick that will work only under Unix. It is similar to one of the answers to this question: Ensuring subprocesses are dead on exiting Python program. The idea is to create a new process group. You can then wait for all processes in the group to terminate.

pid = os.fork()
if pid == 0:
    os.setpgrp()
    oracle_subprocess = subprocess.Popen(OUI_DATABASE_10GR2_SUBPROCESS)
    oracle_subprocess.wait()
    os._exit(0)
else:
    os.waitpid(-pid)

I have not tested this. It creates an extra subprocess to be the leader of the process group, but avoiding that is (I think) quite a bit more complicated.

I found this web page to be helpful as well. http://code.activestate.com/recipes/278731-creating-a-daemon-the-python-way/

回答2:

You can just use os.waitpid with the the pid set to -1, this will wait for all the subprocess of the current process until they finish:

import os
import sys
import subprocess


proc = subprocess.Popen([sys.executable,
                         '-c',
                         'import subprocess;'
                         'subprocess.Popen("sleep 5", shell=True).wait()'])

pid, status = os.waitpid(-1, 0)

print pid, status

This is the result of pstree <pid> of different subprocess forked:

python───python───sh───sleep

Hope this can help :)

回答3:

Check out the following link http://www.oracle-wiki.net/startdocsruninstaller which details a flag you can use for the runInstaller command.

This flag is definitely available for 11gR2, but I have not got a 10g database to try out this flag for the runInstaller packaged with that version.

Regards

回答4:

Everywhere I look seems to say it's not possible to solve this in the general case. I've whipped up a library called 'pidmon' that combines some answers for Windows and Linux and might do what you need.

I'm planning to clean this up and put it on github, possibly called 'pidmon' or something like that. I'll post a link if/when I get it up.

EDIT: It's available at http://github.com/dbarnett/python-pidmon.

I made a special waitpid function that accepts a graft_func argument so that you can loosely define what sort of processes you want to wait for when they're not direct children:

import pidmon
pidmon.waitpid(oracle_subprocess.pid, recursive=True,
        graft_func=(lambda p: p.name == '???' and p.parent.pid == ???))

or, as a shotgun approach, to just wait for any processes started since the call to waitpid to stop again, do:

import pidmon
pidmon.waitpid(oracle_subprocess.pid, graft_func=(lambda p: True))

Note that this is still barely tested on Windows and seems very slow on Windows (but did I mention it's on github where it's easy to fork?). This should at least get you started, and if it works at all for you, I have plenty of ideas on how to optimize it.

来源：https://stackoverflow.com/questions/6102535/python-how-to-determine-subprocess-children-have-all-finished-running