Python subprocess .check_call vs .check_output

Question

My python script (python 3.4.3) calls a bash script via subprocess:

import subprocess as sp
res = sp.check_output(\"bashscript\", shell=True)

Answer 1

check_call() returns as soon as /bin/sh process exits without waiting for descendant processes (assuming shell=True as in your case).

check_output() waits until all output is read. If ssh inherits the pipe then check_output() will wait until it exits (until it closes its inherited pipe ends).

check_call() code example:

#!/usr/bin/env python
import subprocess
import sys
import time

start = time.time()
cmd = sys.executable + " -c 'import time; time.sleep(2)' &"
subprocess.check_call(cmd, shell=True)
assert (time.time() - start) < 1

The output is not read; check_call() returns immediately without waiting for the grandchild background python process.

check_call() is just Popen().wait(). Popen() starts the external process and returns immediately without waiting for it to exit. .wait() collects the exit status for the process -- it doesn't wait for other (grandchildren) processes.

If the output is read (it is redirected and the grandchild python process inherits the stdout pipe):

start = time.time()
subprocess.check_output(cmd, shell=True)
assert (time.time() - start) > 2

then it waits until the background python process that inherited the pipe exits.

check_output() calls Popen().communicate(), to get the output. .communicate() calls .wait() internally i.e., check_output() also waits for the shell to exit and check_output() waits for EOF.

If the grandchild doesn't inherit the pipe then check_output() doesn't wait for it:

start = time.time()
cmd = sys.executable + " -c 'import time; time.sleep(2)' >/dev/null &"
subprocess.check_output(cmd, shell=True)
assert (time.time() - start) < 1

Grandchild's output is redirected to /dev/null i.e., it doesn't inherit the parent's pipe and therefore check_output() may exit without waiting for it.

Note: & at the end which puts the grandchild python process into background. It won't work on Windows where shell=True starts cmd.exe by default.