Real time read from subprocess.stdout on Windows

问题

To emphasize, the problem is real time read instead of non-blocking read. It has been asked before, e.g. subprocess.Popen.stdout - reading stdout in real-time (again). But no satisfactory solution has been proposed.

As an example, the following code tries to simulate the python shell.

import subprocess

p = subprocess.Popen(['python'], stdin=subprocess.PIPE, stdout=subprocess.PIPE)

while True:
    line = input('>>> ')
    p.stdin.write(line.encode())
    print('>>> ', p.stdout.read().decode())

However, it would be blocked when reading from p.stdout. After searching around, I found the following two possible soutions.

using fctrl and O_NONBLOCK
using thread and queue

Whereas the 1st soution may work and only work on linux, the 2nd soution just turn blocking read to non-blocking read, i.e. I cannot get real time output of the subprocess. For example, if I input 'print("hello")', I will get nothing from p.stdout using 2nd solution.

Perhaps, someone would suggest p.communite. Unfortunately, it is not suitable in this case, since it would close stdin as described here.

So, is there any solutions for Windows?

Edited: Even if -u is turned on and p.stdout.read is replaced with p.stdout.readline, the problem still exists.

import subprocess

p = subprocess.Popen(['python', '-u'], stdin=subprocess.PIPE, stdout=subprocess.PIPE)

while True:
    line = input('>>> ')
    p.stdin.write(line.encode())
    p.stdin.flush()
    print('>>> ', p.stdout.readline().decode())

Solution: The following is the final code based on J.F. Sebastian's answer and comments.

from subprocess import Popen, PIPE, STDOUT

with Popen(
        ['python', '-i', '-q'],
        stdin=PIPE, stdout=PIPE, stderr=STDOUT,
        bufsize=0
    ) as process:
    while True:
        line = input('>>> ')
        if not line:
            break
        process.stdin.write((line+'\n').encode())
        print(process.stdout.readline().decode(), end='')

It should be noted that the program would hang when the command triggers no output.

回答1:

Here's a complete working example that uses a subprocess interactively:

#!/usr/bin/env python3
import sys
from subprocess import Popen, PIPE, DEVNULL

with Popen([sys.executable, '-i'], stdin=PIPE, stdout=PIPE, stderr=DEVNULL,
           universal_newlines=True) as process:
    for i in range(10):
        print("{}**2".format(i), file=process.stdin, flush=True)
        square = process.stdout.readline()
        print(square, end='')

Here's another example: how to run [sys.executable, '-u', 'test.py'] interactively.

来源：https://stackoverflow.com/questions/36879695/real-time-read-from-subprocess-stdout-on-windows

标签

python

windows

subprocess

output-buffering

io-buffering