Making python wait until subprocess.call has finished its command

问题

So this question is following on from this (Read comments aswell as that is the path I took). I just want the first call robocopy to finish executing before moving on to the rest of the code. As I want the second robocopy to just skip all of the files as they have already been copied over. However what is happening is that the rest of the script will run (ie starts the second robocopy) while the first robocopy is copying over the files. Below is the code:

call(["start", "cmd", "/K", "RoboCopy.exe", f"{self.srcEntry.get()}", f"{self.dstEntry.get()}", "*.*", "/E", "/Z", "/MT:8"], stdout=PIPE, shell=True) 
temp2 = Popen(["RoboCopy.exe", f"{self.srcEntry.get()}", f"{self.dstEntry.get()}", "*.*", "/E", "/Z"], stdout=PIPE, stdin=PIPE, shell=True)

EDIT 1:

Issue is noticeable when copying over large files. I'm thinking about putting in a sleep function which is dependent upon the total size of the files to be copied over. However this doesn't take into account upload/download speeds as the files will be transferred over a network.

回答1:

I use the following function to launch my commands which waits until the action has finished, but has a timeout:

import os
import logging

logger = logging.getLogger()

def command_runner(command, valid_exit_codes=None, timeout=30, shell=False, decoder='utf-8'):
    """
    command_runner 2019103101
    Whenever we can, we need to avoid shell=True in order to preseve better security
    Runs system command, returns exit code and stdout/stderr output, and logs output on error
    valid_exit_codes is a list of codes that don't trigger an error
    """

    try:
        # universal_newlines=True makes netstat command fail under windows
        # timeout does not work under Python 2.7 with subprocess32 < 3.5
        # decoder may be unicode_escape for dos commands or utf-8 for powershell
        if sys.version_info >= (3, 0):
            output = subprocess.check_output(command, stderr=subprocess.STDOUT, shell=shell,
                                             timeout=timeout, universal_newlines=False)
        else:
            output = subprocess.check_output(command, stderr=subprocess.STDOUT, shell=shell,
                                             universal_newlines=False)
        output = output.decode(decoder, errors='backslashreplace')
    except subprocess.CalledProcessError as exc:
        exit_code = exc.returncode
        # noinspection PyBroadException
        try:
            output = exc.output
            try:
                output = output.decode(decoder, errors='backslashreplace')
            except Exception as subexc:
                logger.debug(subexc, exc_info=True)
                logger.debug('Cannot properly decode error. Text is %s' % output)
        except Exception:
            output = "command_runner: Could not obtain output from command."
        if exit_code in valid_exit_codes if valid_exit_codes is not None else [0]:
            logger.debug('Command [%s] returned with exit code [%s]. Command output was:' % (command, exit_code))
            if output:
                logger.debug(output)
            return exc.returncode, output
        else:
            logger.error('Command [%s] failed with exit code [%s]. Command output was:' %
                         (command, exc.returncode))
            logger.error(output)
            return exc.returncode, output
    # OSError if not a valid executable
    except OSError as exc:
        logger.error('Command [%s] faild because of OS [%s].' % (command, exc))
        return None, exc
    except subprocess.TimeoutExpired:
        logger.error('Timeout [%s seconds] expired for command [%s] execution.' % (timeout, command))
        return None, 'Timeout of %s seconds expired.' % timeout
    except Exception as exc:
        logger.error('Command [%s] failed for unknown reasons [%s].' % (command, exc))
        logger.debug('Error:', exc_info=True)
        return None, exc
    else:
        logger.debug('Command [%s] returned with exit code [0]. Command output was:' % command)
        if output:
            logger.debug(output)
        return 0, output


# YOUR CODE HERE

executable = os.path.join(os.environ['SYSTEMROOT'], 'system32', 'robocopy.exe')
mycommand = '"%s" "%s" "%s" "%s"' % (executable, source, dest, options)
result, output = command_runner(mycommand, shell=True)

回答2:

What I've found out: Thanks to QuantumChris. I've found out that robocopy returns from the terminal and back into my script although I've used subprocess.run which should have paused my script until it had finished running. I'm stalling the second robocopy from running by checking if the files have been copied over to the destination folder before progressing with the second robocopy. The issue is that if the last file is big then os.path.isfile() detects the file WHILE it is still being copied over. So it engages the second robocopy however the second robocopy doesn't detect that last file and so attempts to copy the file over but recognises that it can't access the file as it is already in use (by the first robocopy) so it waits 30 secs before trying again. After the 30 secs it is able to access the file and copies it over. What I would like to do now is to make my last file an empty dummy file which I don't care about it being copied twice as it is empty. Robocopy seems to copy over the files according to ASCII order. So I've named the file ~~~~~.txt :D

回答3:

Try:

while temp2.poll() is not None:
    # ... do something else, sleep, etc

out, err = temp2.communicate()

来源：https://stackoverflow.com/questions/60224032/making-python-wait-until-subprocess-call-has-finished-its-command

标签

python

subprocess

call

popen

robocopy