Using subprocess with select and pty hangs when capturing output

Question

I\'m trying to write a python program that is able to interact with other programs. That means sending stdin and receiving stdout data. I cannot use pexpect (although it definit

Answer 1

There are a number of things you can change to make your code correct. The simplest thing I can think of is just to close your parent process's copy of the slave fd after forking, so that when the child exits and closes its own slave fd, the parent's select.select() will mark the master as available for read, and the subsequent os.read() will give an empty result and your program will complete. (The pty master won't see the slave end as being closed until both copies of the slave fd are closed.)

So, just one line:

os.close(slave)

..placed immediately after the subprocess.Popen call, ought to fix your problem.

However, there are possibly better answers, depending on exactly what your requirements are. As someone else noted, you don't need a pty just to avoid buffering. You could use a bare os.pipe() in place of pty.openpty() (and treat the return value exactly the same). A bare OS pipe will never buffer; if the child process isn't buffering its output, then your select() and os.read() calls won't see buffering either. You would still need the os.close(slave) line, though.

But it's possible that you do need a pty for different reasons. If some of your child programs expect to be run interactively much of the time, then they might be checking to see if their stdin is a pty and behaving differently depending on the answer (lots of common utilities do this). If you really do want the child to think it has a terminal allocated for it, then the pty module is the way to go. Depending on how you'll run runner.py, you may need to switch from using subprocess to pty.fork(), so that the child has its session ID set and the pty pre-opened (or see the source for pty.py to see what it does and duplicate the appropriate parts in your subprocess object's preexec_fn callable).