Python using STDIN in child Process

Question

So I have a program, in the \"main\" process I fire off a new Process object which (what I want) is to read lines from STDIN and append them to a Queue object.

Ess

Answer 1

You could use threading and keep it all on the same process:

from multiprocessing import Queue
from Queue import Empty
from threading import Thread

def sub_proc(q):
    some_str = ""
    while True:
        some_str = raw_input("> ")
        if some_str.lower() == "quit":
            return
        q.put_nowait(some_str)

if __name__ == "__main__":
    q = Queue()
    qproc = Thread(target=sub_proc, args=(q,))
    qproc.start()
    qproc.join()

    while True:
        try:
            print q.get(False)
        except Empty:
            break

Answer 2

I solved a similar issue by passing the original stdin file descriptor to the child process and re-opening it there.

def sub_proc(q,fileno):
    sys.stdin = os.fdopen(fileno)  #open stdin in this process
    some_str = ""
    while True:
        some_str = raw_input("> ")

        if some_str.lower() == "quit":
            return
        q.put_nowait(some_str)

if __name__ == "__main__":
    q = Queue()
    fn = sys.stdin.fileno() #get original file descriptor
    qproc = Process(target=sub_proc, args=(q,fn))
    qproc.start()
    qproc.join()

This worked for my relatively simple case. I was even able to use the readline module on the re-opened stream. I don't know how robust it is for more complex systems.

Answer 3

In short, the main process and your second process don't share the same STDIN.

from multiprocessing import Process, Queue
import sys

def sub_proc():
    print sys.stdin.fileno()

if __name__ == "__main__":
    print sys.stdin.fileno()
    qproc = Process(target=sub_proc)
    qproc.start()
    qproc.join()

Run that and you should get two different results for sys.stdin.fileno()

Unfortunately, that doesn't solve your problem. What are you trying to do?

Answer 4

If you don't want to pass stdin to the target processes function, like in @Ashelly's answer, or just need to do it for many different processes, you can do it with multiprocessing.Pool via the initializer argument:

import os, sys, multiprocessing

def square(num=None):
    if not num:
        num = int(raw_input('square what? ')) 
    return num ** 2

def initialize(fd):
    sys.stdin = os.fdopen(fd)

initargs = [sys.stdin.fileno()]
pool = multiprocessing.Pool(initializer=initialize, initargs=initargs)
pool.apply(square, [3])
pool.apply(square)

the above example will print the number 9, followed by a prompt for input and then the square of the input number.

Just be careful not to have multiple child processes reading from the same descriptor at the same time or things may get... confusing.