Python subprocess module: How to reopen PIPE?

问题

I have a piece of code that needs to run node js. In order to write to the file for node js to recieve the command, I use the following code:

def execute(comm):
    file_number = proc.stdin.fileno()
    file = open(file_number, "w+")
    file.write(comm)
    file.close()

execute("""console.log("hi");""")
execute("""console.log("bye");""")

However, when I run execute a second time, I get the following error:

OSError: [WinError 6] The handle is invalid

What does this error mean and how do I fix it? Here is the full code:

import subprocess
import threading
import time

class nodePrompt:
    def __init__(self):
        self.proc = subprocess.Popen("node", shell=True, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, universal_newlines=True)
        self.on_out_func = print
        self.on_out_args=[]
        self.on_out_kwargs={}
        self.on_err_func = print
        self.on_err_args=[]
        self.on_err_kwargs={}
        self.on_out_thread=None
        self.on_err_thread=None

    def execute(self, comm):
        file_number = self.proc.stdin.fileno()
        file = open(file_number, "w+")
        file.write(comm)
        file.close()


    def on_out(function, *args, **kwargs):
        self.on_out_func=function
        self.on_out_args=args
        self.on_out_kwargs=kwargs

    def on_err(function, *args, **kwargs):
        self.on_err_func=function
        self.on_err_args=args
        self.on_err_kwargs=kwargs

    def on_out_handle(self):
        for i in iter(self.proc.stdout.readline, ""):
            self.on_out_func(i, *self.on_out_args, **self.on_out_kwargs)

    def start_on_out(self, daemon=False):
        self.on_out_thread=threading.Thread(target=self.on_out_handle)
        self.on_out_thread.setDaemon(daemon)
        self.on_out_thread.start()

    def on_err_handle(self):
        for i in iter(self.proc.stderr.readline, ""):
            self.on_err_func(i, *self.on_err_args, **self.on_err_kwargs)

    def start_on_err(self, daemon=False):
        self.on_err_thread=threading.Thread(target=self.on_err_handle)
        self.on_err_thread.setDaemon(daemon)
        self.on_err_thread.start()

prompt = nodePrompt()
prompt.start_on_out()
prompt.start_on_err()
prompt.execute("console.log(\"HI\");")
time.sleep(1)
prompt.execute("console.log(\"HI\");")

回答1:

A closed FIFO can't be re-opened. All you can do is not close it until you're really done. Use `flush()` instead to make sure writes happen immediately.

This is a problem with node.js not running code incrementally (or not flushing its output) when its input and output go to pipelines. Python is provably doing the right thing, after fixing your code to work with self.proc.stdin directly.

That is, after changing execute to be defined as:

    def execute(self, comm):
        self.proc.stdin.write(comm)
        self.proc.stdin.flush()

...we can monitor the activity of both the Python interpreter and the copy of node.js it starts.

Unfortunately, `node` doesn't behave the way you want it to.

Pay attention to the timestamps of the below trace, which used sysdig to track operations during your program's execution:

5178 18:55:37.047606506 4 python (18260) < write res=19 data=console.log("HI");. 
8239 18:55:37.095849210 1 node (18261) < read res=19 data=console.log("HI");. 
8590 18:55:38.048679102 4 python (18260) < write res=19 data=console.log("HI");. 
8595 18:55:38.048710436 4 python (18260) < close res=0 
8597 18:55:38.048742124 1 node (18261) < read res=19 data=console.log("HI");. 
8603 18:55:38.048911687 1 node (18261) < read res=0 data= 
8633 18:55:38.051116830 1 node (18261) < write res=3 data=HI. 
8634 18:55:38.051158022 6 python (18262) < read res=3 data=HI. 
8636 18:55:38.051199286 6 python (18262) < write res=3 data=HI. 
8642 18:55:38.051400907 1 node (18261) < write res=3 data=HI. 
8643 18:55:38.051441455 6 python (18262) < read res=3 data=HI. 
8645 18:55:38.051459654 6 python (18262) < write res=3 data=HI.

Python writes the first console.log("HI"), and Node reads it, long before Node writes any output at all.

I'd suggest looking into whether Node runs code as it's received when it comes from a non-TTY source, or if it tries to read the entire stream to its end and parse it before running anything.

A Suggested Reproducer

If you're going to ask this question for a node audience, consider giving them the following bash script:

{
  echo 'console.log("HI");'
  sleep 1
  echo 'console.log("HI");'
  sleep 1
  echo "Already sent both commands to node" >&2
} | node

Its output is:

Already sent both commands to node
HI
HI

...whereas one might hope for at least one HI to be printed earlier, since there's a full two seconds' delay after the first command is sent.

回答2:

try adding file.close_proc() under file.close()

来源：https://stackoverflow.com/questions/61035980/python-subprocess-module-how-to-reopen-pipe

标签

python

subprocess