Using module 'subprocess' with timeout

后端 未结 29 2374
旧巷少年郎
旧巷少年郎 2020-11-21 15:15

Here\'s the Python code to run an arbitrary command returning its stdout data, or raise an exception on non-zero exit codes:

proc = subprocess.P         


        
相关标签:
29条回答
  • 2020-11-21 16:15

    I had the problem that I wanted to terminate a multithreading subprocess if it took longer than a given timeout length. I wanted to set a timeout in Popen(), but it did not work. Then, I realized that Popen().wait() is equal to call() and so I had the idea to set a timeout within the .wait(timeout=xxx) method, which finally worked. Thus, I solved it this way:

    import os
    import sys
    import signal
    import subprocess
    from multiprocessing import Pool
    
    cores_for_parallelization = 4
    timeout_time = 15  # seconds
    
    def main():
        jobs = [...YOUR_JOB_LIST...]
        with Pool(cores_for_parallelization) as p:
            p.map(run_parallel_jobs, jobs)
    
    def run_parallel_jobs(args):
        # Define the arguments including the paths
        initial_terminal_command = 'C:\\Python34\\python.exe'  # Python executable
        function_to_start = 'C:\\temp\\xyz.py'  # The multithreading script
        final_list = [initial_terminal_command, function_to_start]
        final_list.extend(args)
    
        # Start the subprocess and determine the process PID
        subp = subprocess.Popen(final_list)  # starts the process
        pid = subp.pid
    
        # Wait until the return code returns from the function by considering the timeout. 
        # If not, terminate the process.
        try:
            returncode = subp.wait(timeout=timeout_time)  # should be zero if accomplished
        except subprocess.TimeoutExpired:
            # Distinguish between Linux and Windows and terminate the process if 
            # the timeout has been expired
            if sys.platform == 'linux2':
                os.kill(pid, signal.SIGTERM)
            elif sys.platform == 'win32':
                subp.terminate()
    
    if __name__ == '__main__':
        main()
    
    0 讨论(0)
  • 2020-11-21 16:17

    Here is Alex Martelli's solution as a module with proper process killing. The other approaches do not work because they do not use proc.communicate(). So if you have a process that produces lots of output, it will fill its output buffer and then block until you read something from it.

    from os import kill
    from signal import alarm, signal, SIGALRM, SIGKILL
    from subprocess import PIPE, Popen
    
    def run(args, cwd = None, shell = False, kill_tree = True, timeout = -1, env = None):
        '''
        Run a command with a timeout after which it will be forcibly
        killed.
        '''
        class Alarm(Exception):
            pass
        def alarm_handler(signum, frame):
            raise Alarm
        p = Popen(args, shell = shell, cwd = cwd, stdout = PIPE, stderr = PIPE, env = env)
        if timeout != -1:
            signal(SIGALRM, alarm_handler)
            alarm(timeout)
        try:
            stdout, stderr = p.communicate()
            if timeout != -1:
                alarm(0)
        except Alarm:
            pids = [p.pid]
            if kill_tree:
                pids.extend(get_process_children(p.pid))
            for pid in pids:
                # process might have died before getting to this line
                # so wrap to avoid OSError: no such process
                try: 
                    kill(pid, SIGKILL)
                except OSError:
                    pass
            return -9, '', ''
        return p.returncode, stdout, stderr
    
    def get_process_children(pid):
        p = Popen('ps --no-headers -o pid --ppid %d' % pid, shell = True,
                  stdout = PIPE, stderr = PIPE)
        stdout, stderr = p.communicate()
        return [int(p) for p in stdout.split()]
    
    if __name__ == '__main__':
        print run('find /', shell = True, timeout = 3)
        print run('find', shell = True)
    
    0 讨论(0)
  • 2020-11-21 16:17

    for python 2.6+, use gevent

     from gevent.subprocess import Popen, PIPE, STDOUT
    
     def call_sys(cmd, timeout):
          p= Popen(cmd, shell=True, stdout=PIPE)
          output, _ = p.communicate(timeout=timeout)
          assert p.returncode == 0, p. returncode
          return output
    
     call_sys('./t.sh', 2)
    
     # t.sh example
     sleep 5
     echo done
     exit 1
    
    0 讨论(0)
  • 2020-11-21 16:18

    Although I haven't looked at it extensively, this decorator I found at ActiveState seems to be quite useful for this sort of thing. Along with subprocess.Popen(..., close_fds=True), at least I'm ready for shell-scripting in Python.

    0 讨论(0)
  • 2020-11-21 16:18

    Was just trying to write something simpler.

    #!/usr/bin/python
    
    from subprocess import Popen, PIPE
    import datetime
    import time 
    
    popen = Popen(["/bin/sleep", "10"]);
    pid = popen.pid
    sttime = time.time();
    waittime =  3
    
    print "Start time %s"%(sttime)
    
    while True:
        popen.poll();
        time.sleep(1)
        rcode = popen.returncode
        now = time.time();
        if [ rcode is None ]  and  [ now > (sttime + waittime) ] :
            print "Killing it now"
            popen.kill()
    
    0 讨论(0)
提交回复
热议问题