Wrap subprocess' stdout/stderr

Question

I\'d like to both capture and display the output of a process that I invoke through Python\'s subprocess.

I thought I could just pass my file-like object as named pa

Answer 1

Stdin, stdout and stderr of a process need to be real file descriptors. (That is actually not a restriction imposed by Python, but rather how pipes work on the OS level.) So you will need a different solution.

If you want to track both stdout an stderr in real time, you will need asynchronous I/O or threads.

• Asynchronous I/O: With the standard synchronous (=blocking) I/O, a read to one of the streams could block, disallowing access to the other one in real time. If you are on Unix, you can use non-blocking I/O as described in this answer. However, on Windows you will be out of luck with this approach. More on asynchronous I/O in Python and some alternatives are shown in this video.

• Threads: Another common way to deal with this problem is to create one thread for each file descriptor you want to read from in real time. The threads only handle the file descriptor they are assinged to, so blocking I/O won't harm.

Answer 2

A file-like is not close enough. It must be an actual file with an actual file descriptor. Use subprocess's support for pipes and read from them as appropriate.

Answer 3

Have a look here.

p = subprocess.Popen(cmd,
                 shell=True,
                 bufsize=64,
                 stdin=subprocess.PIPE,
                 stderr=subprocess.PIPE,
                 stdout=subprocess.PIPE)

Answer 4

Is there a reason you have a class inside a class? And stdout and stderr can take any file like line for example try. So simply passing an open file type or stringIO should be enough to modify the stream

import sys
sys.stdout = open('test.txt','w')
print "Testing!"
sys.stdout.write('\nhehehe')
sys.stdout = sys.__stdout__
sys.exit(0)