Python subprocess readlines() hangs

Question

The task I try to accomplish is to stream a ruby file and print out the output. (NOTE: I don\'t want to print out everything at once)

Answer 1

Basically what you are looking at here is a race condition between your proc.poll() and your readline(). Since the input on the master filehandle is never closed, if the process attempts to do a readline() on it after the ruby process has finished outputting, there will never be anything to read, but the pipe will never close. The code will only work if the shell process closes before your code tries another readline().

Here is the timeline:

readline()
print-output
poll()
readline()
print-output (last line of real output)
poll() (returns false since process is not done)
readline() (waits for more output)
(process is done, but output pipe still open and no poll ever happens for it).

Easy fix is to just use the subprocess module as it suggests in the docs, not in conjunction with openpty:

http://docs.python.org/library/subprocess.html

Here is a very similar problem for further study:

Using subprocess with select and pty hangs when capturing output