How do I close the stdout-pipe when killing a process started with python subprocess Popen?

Question

I wonder if it is possible to shut down the communication pipe when killing a subprocess started in a different thread. If I do not call communicate() then kill() will work as e

Answer 1

It looks like you may be a victim of Python's super coarse grained concurrency. Change your script to this:

#!/bin/bash
echo "sleeping five"
sleep 5
echo "sleeping five again"
sleep 5
echo "slept five"

And then the output becomes:

process: sleeping five

real    0m5.134s
user    0m0.000s
sys     0m0.010s

If the entire script ran, the time would be 10s. So it looks like the python control thread doesn't actually run until after the bash script sleeps. Similarly, if you change your script to this:

#!/bin/bash
echo "sleeping five"
sleep 1
sleep 1
sleep 1
sleep 1
sleep 1
echo "slept five"

Then the output becomes:

process: sleeping five

real    0m1.150s
user    0m0.010s
sys     0m0.020s

In short, your code works as logically implemented. :)

Answer 2

I think the problem is that process.kill() only kills the immediate child process (bash), not the sub-processes of the bash script.

The problem and solution are described here:

• http://www.doughellmann.com/PyMOTW/subprocess/#process-groups-sessions
• How to terminate a python subprocess launched with shell=True

Use Popen(..., preexec_fn=os.setsid) to create a process group and os.pgkill to kill the entire process group. eg

import os
import signal
import subprocess
import time
from threading import Thread

process = None

def executeCommand(command, runCommand):
    Thread(target=runCommand, args=(command,)).start()

def runCommand(command):
    global process
    args = command.strip().split()
    process = subprocess.Popen(
        args, shell=False, stdout=subprocess.PIPE, preexec_fn=os.setsid)

    for line in process.communicate():
        if line:
            print "process:", line,

if __name__ == '__main__':
    executeCommand("./ascript.sh", runCommand)
    time.sleep(1)
    os.killpg(process.pid, signal.SIGKILL)

---

$ time python poc.py 
process: sleeping five

real    0m1.051s
user    0m0.032s
sys 0m0.020s

Answer 3

It seems to me that the easiest way to do this and sidestep the multithreading problems would be to set a kill flag from the main thread, and check for it in the script-running thread just before the communication, killing the script when the flag is True.