Custom Popen.communicate method gives wrong output

问题

Let's start by considering this code:

proc_stdin.py

import sys

if __name__ == '__main__':
    for i, line in enumerate(sys.stdin):
        sys.stdout.write(line)

test.py

import subprocess


def run_bad(target, input=None):
    proc = subprocess.Popen(
        target,
        universal_newlines=True,
        shell=True,
        stderr=subprocess.STDOUT,
        stdin=subprocess.PIPE if input else subprocess.DEVNULL,
        stdout=subprocess.PIPE,
    )

    if input:
        proc.stdin.write(input)
        proc.stdin.flush()
        proc.stdin.close()

    lines = []
    for line in iter(proc.stdout.readline, ""):
        line = line.rstrip("\n")
        lines.append(line)
    proc.stdout.close()

    ret_code = proc.wait()
    return "\n".join(lines)


def run_good(target, input):
    return subprocess.Popen(
        target,
        universal_newlines=True,
        shell=True,
        stderr=subprocess.STDOUT,
        stdin=subprocess.PIPE,
        stdout=subprocess.PIPE,
    ).communicate(input=input)[0]


if __name__ == '__main__':
    lst = [
        "",
        "token1",
        "token1\n",
        "token1\r\n",
        "token1\n\n",
        "token1\r\n\ntoken2",
        "token1 token2",
        "token1\ntoken2",
        "token1\r\ntoken2",
        "token1\n\ntoken2",
        "token1\r\n\ntoken2",
        "token1 \ntoken2\ntoken2\n"
    ]
    cmd = "python proc_stdin.py"

    for inp in lst:
        a, b = run_bad(cmd, inp), run_good(cmd, inp)
        if a != b:
            print("Error: {} vs {}".format(repr(a), repr(b)))
        else:
            print("ok: {}".format(repr(a)))

Output:

ok: ''
ok: 'token1'
Error: 'token1' vs 'token1\n'
Error: 'token1\n' vs 'token1\n\n'
Error: 'token1\n' vs 'token1\n\n'
ok: 'token1\n\n\ntoken2'
ok: 'token1 token2'
ok: 'token1\ntoken2'
ok: 'token1\n\ntoken2'
ok: 'token1\n\ntoken2'
ok: 'token1\n\n\ntoken2'
Error: 'token1 \ntoken2\ntoken2' vs 'token1 \ntoken2\ntoken2\n'

My question is, why is the output of both run_bad & run_good not equal in all cases? How would you change the run_bad function so the output becomes equal than run_good?

You also may wonder, why are you not using directly Popen.communicate for this particular case or other helpers from subprocess module? Well, in the real world case I'm creating a plugin for SublimeText3 which is forcing me to stick to python3.3 (can't use many of the modern subprocess goodies) plus I'd like to inject some callbacks while reading the lines from stdout and that's something I can't do by using the Popen.communicate method (as far as I know).

Thanks in advance.

回答1:

If you strip newlines from every line and then add them back between the lines, what happens to the last newline (if any)? (There’s no final, empty line after a final newline because your iter discards it.) This is why Python’s readline (or line iteration) function includes the newlines: they’re necessary to represent the end of the file accurately.

来源：https://stackoverflow.com/questions/60858329/custom-popen-communicate-method-gives-wrong-output

标签

python

subprocess

python-3.3