Saving stdout from subprocess.Popen to file line by line

Question

My python script uses subprocess to call an another script, which produces output very slow(line-by-line basis). I would like to write the output line by line to file not when t

Answer 1

You can interact with the process using poll so that you can attempt to interact with it line by line:

For example:

process = subprocess.Popen(["ls", "-lart"],
                 bufsize=-1, # fully buffered (default)
                 stdin=subprocess.PIPE,
                 stdout=subprocess.PIPE,
                 stderr=subprocess.PIPE,
                 cwd=os.curdir,
                 env=os.environ)
my_stdout_file = open("stdout.txt", "w")
while True:
    process.poll()
    line = process.stdout.readline()
    my_stdout_file.write(line)
    eline = process.stderr.readline()
    if line:
        stdout_lines.append(line)
    if eline:
        stderr_lines.append(eline)
    if (line == "" and eline == "" and
        process.returncode != None):
        break