Preserving bash redirection in a python subprocess

Question

To begin with, I am only allowed to use python 2.4.4

I need to write a process controller in python which launches and various subprocesses monitors how they affect the

Answer 1

If I understand your question correctly, it sounds like what you are looking for here is to be able to launch a list of scripts with the output redirected to files. In that case, launch each of your tasks something like this:

task = subprocess.Popen(['python', 'myscript', 'arg1', 'arg2', 'arg3'],
    stdout=open('output.log', 'w'), stderr=open('err.log', 'w'))

Doing this means that the subprocess's stdout and stderr are redirected to files that the monitoring process opened, but the monitoring process does not have to be involved in copying data around. You can also redirect the subprocess stdins as well, if needed.

Note that you'll likely want to handle error cases and such, which aren't handled in this example.

Answer 2

You can use existing file descriptors as the stdout/stderr arguments to subprocess.Popen. This should be exquivalent to running from with redirection from bash. That redirection is implemented with fdup(2) after fork and the output should never touch your program. You can probably also pass fopen('/dev/null') as a file descriptor.

Alternatively you can redirect the stdout/stderr of your controller program and pass None as stdout/stderr. Children should print to your controllers stdout/stderr without passing through python itself. This works because the children will inherit the stdin/stdout descriptors of the controller, which were redirected by bash at launch time.

Answer 3

The subprocess module is good.

You can also do this on *ix with os.fork() and a periodic os.wait() with a WNOHANG.