How to get subprocess stdout while running git command?

问题

I have a program written in python and used git command in it.. For some reason I don't want to use git-python or others instead of subprocess. But I'm currently stuck in getting git clone output.

I've tried some code snippet. Some works fine with commands like ping 8.8.8.8, but not the git clone.

for example

using thread

def log_worker(stdout):
    while True:
        last = non_block_read(stdout).strip() 
        if last != "":
            print(last)


def non_block_read(output):
    fd = output.fileno()
    fl = fcntl.fcntl(fd, fcntl.F_GETFL)
    fcntl.fcntl(fd, fcntl.F_SETFL, fl | os.O_NONBLOCK)
    try:
        return output.read()
    except:
        return ''

def test():
    mysql_process = subprocess.Popen(
        "ping google.com",
        shell=True,
        stdout=subprocess.PIPE,
        stderr=subprocess.PIPE)

    thread = Thread(target=log_worker, args=[mysql_process.stdout])
    thread.daemon = True
    thread.start()

    mysql_process.wait()
    thread.join(timeout=1)

test()

newlines = ['\n', '\r\n', '\r']
def unbuffered(proc, stream='stdout'):
    stream = getattr(proc, stream)
    with contextlib.closing(stream):
        while True:
            print('tt')
            out = []
            last = stream.read(1)
            # Don't loop forever
            if last == '' and proc.poll() is not None:
                break
            print('last', last)
            while last not in newlines:
                print("loop")
                # Don't loop forever
                if last == '' and proc.poll() is not None:
                    break
                out.append(last)
                last = stream.read(1)
            out = ''.join(out)
            yield out

def example():
    cmd = ['ls', '-l', '/']
    proc = subprocess.Popen(
        cmd,
        stdout=subprocess.PIPE,
        stderr=subprocess.STDOUT,
        # Make all end-of-lines '\n'
        universal_newlines=True,
        shell = True
    )
    for line in unbuffered(proc):
        print('new line')
        print line

example()

and most common one

for line in iter(proc.stdout.readline, ''):
    sys.stdout.write('{:.2f}  {}\n'.format(
        time.time() - start,
        line.rstrip()
    ))
    sys.stdout.flush()

all of them works fine with ping google.com, but not git clone. Is there any way to solve this? Thanks in advance!

UPDATE1: In face, I'm just want to get the finished percent of git clone. Log or any log files are not needed.

回答1:

When not writing to a terminal, git clone doesn't have any output to either stdout or stderr, except on error.

When writing to a terminal, of course, it has lots of output—but that output is progress bars that are continually overwritten. Usually, you don't want that—it's going to be a big mess of control characters and repeated lines.

But if you do want it, there are two options.

First, you can use a PTY (Pseudo-TTY). You can create a PTY with os.openpty, then hand the PTY off explicitly to the child process. Or you can use os.forkpty, which handles forking and automatically hooking up the PTY so all you have to do is call one of the os.exec functions. Or you can use the pty module. (It's not entirely clear which is more portable; openpty and forkpty claim that pty is more portable, and conceptually it's designed that way… but it's also only really tested on Linux.)

Note that git wants the PTY as its stderr, not its stdout.

Alternatively, most git commands have a --progress flag that causes them to write progress to stderr even if it's not a terminal. At least as of the version documented here, this includes clone, but of course you should check the man for your local version. So, that may be all you need. (Also see the --verbose flag.)

However, this may not be as nice. For me, when I provide a PTY with no attached termcaps, I get each line followed by a \r without \n to overwrite it; when I use the --progress option, git detects the termcaps of whatever terminal my script happens to be running it, which means I end up getting ANSI color codes as well as \rs.

Of course either way I'm getting hundreds of useless lines that I have no interest in, but I assume that's what you wanted? (Or maybe you want to use universal_newlines='\r' to translate the '\r' to '\n'? That's slightly cheating, because this is self-overwriting Unix terminal output, and you're pretending it's classic-Mac output… but it works.)

来源：https://stackoverflow.com/questions/25986533/how-to-get-subprocess-stdout-while-running-git-command

标签

python

git

subprocess