So I noticed subprocess.call
while it waits for the command to finish before proceeding with the python script, I have no way of getting the stdout, except with
I'd try something like:
#!/usr/bin/python
from __future__ import print_function
import shlex
from subprocess import Popen, PIPE
def shlep(cmd):
'''shlex split and popen
'''
parsed_cmd = shlex.split(cmd)
## if parsed_cmd[0] not in approved_commands:
## raise ValueError, "Bad User! No output for you!"
proc = Popen(parsed_command, stdout=PIPE, stderr=PIPE)
out, err = proc.communicate()
return (proc.returncode, out, err)
... In other words let shlex.split() do most of the work. I would NOT attempt to parse the shell's command line, find pipe operators and set up your own pipeline. If you're going to do that then you'll basically have to write a complete shell syntax parser and you'll end up doing an awful lot of plumbing.
Of course this raises the question, why not just use Popen with the shell=True (keyword) option? This will let you pass a string (no splitting nor parsing) to the shell and still gather up the results to handle as you wish. My example here won't process any pipelines, backticks, file descriptor redirection, etc that might be in the command, they'll all appear as literal arguments to the command. Thus it is still safer then running with shell=True ... I've given a silly example of checking the command against some sort of "approved command" dictionary or set --- through it would make more sense to normalize that into an absolute path unless you intend to require that the arguments be normalized prior to passing the command string to this function.
I am using the following construct, although you might want to avoid shell=True
. This gives you the output and error message for any command, and the error code as well:
process = subprocess.Popen(cmd, shell=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
# wait for the process to terminate
out, err = process.communicate()
errcode = process.returncode
subprocess.check_output(...)
calls the process, raises if its error code is nonzero, and otherwise returns its stdout. It's just a quick shorthand so you don't have to worry about PIPE
s and things.
If your process gives a huge stdout and no stderr, communicate()
might be the wrong way to go due to memory restrictions.
Instead,
process = subprocess.Popen(cmd, shell=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
# wait for the process to terminate
for line in process.stdout: do_something(line)
errcode = process.returncode
might be the way to go.
process.stdout
is a file-like object which you can treat as any other such object, mainly:
read()
from itreadline()
from it andThe latter is what I do above in order to get its contents line by line.
With Python 3.8 this workes for me. For instance to execute a python script within the venv:
import subprocess
import sys
res = subprocess.run([
sys.executable, # venv3.8/bin/python
'main.py', '--help',],
stdout=PIPE,
text=True)
print(res.stdout)