How to create non-blocking continuous reading from `stdin`?

Question

I have a single process, which has been created like this:

p = subprocess.Popen(args   = \'./myapp\',
                     stdin  = subprocess.PIPE,
                     


        

Answer 1

Trying to investigate your program, I wrote my own "continually stream stuff to cat and catch what it returns" program. I didn't implement the subprocess side of it, but hopefully the structure is similar.

This line is very odd about your program...

for line in iter(sys.stdin.readline, ''):
    q.put(line)
sys.stdin.close()

That looks an awful lot like

for line in stdin:
    q.put(line)

Note that the loop is going to end when the pipe is closed and there's no need to re-close it afterwards.

If you need to continously asynchronously read stdin, you should be able to construct a reading thread near-identical to child_reader in the code below. Just replace child.stdout with stdin.

import subprocess
import threading
import random

# We may need to guard this?
child = subprocess.Popen('cat', stdout=subprocess.PIPE, stdin=subprocess.PIPE)

# Continuously print what the process outputs...
def print_child():
    for line in child.stdout:
        print(line)

child_reader = threading.Thread(target = print_child)
child_reader.start()

for i in range(10000):
    chars = 'ABC\n'
    child.stdin.write(random.choice(chars).encode())

# Send EOF.
# This kills the cat.
child.stdin.close()

# I don't think order matters here?
child.wait()
child_reader.join()