Python subprocess running in background before returning output

Question

I have some Python code that I want to debug with perf. For that purpose I want to use subprocess. The following command returns instruction-related information of a process

Answer 1

First: I advise against calling perf from within your python process (as you see in the complexity of the task below), but instead use is from the command line:

sudo perf stat -- python test.py

If you really want to call perf from within python then you'll face some tricky problems:

1. to terminate perf and make it output the gathered performance stats you need to send it the SIGINT signal (try it out with sudo perf stat -p mypid: ctrl-\ will print nothing whereas ctrl-c will)
2. you need to capture stderr as perf sends its output to stderr (at least in my version)
3. you need to use fork() with one process sending SIGINT and the other reading it's output while the process dies. Without forks it won't work because after you SIGINTed the perf process you cannot read from stdin any more as the process is already gone, and when you read from stdin first you won't get any output until perf is correctly terminated.

That means you'd end up with this python program:

import subprocess
import os
import signal
import time

perf = subprocess.Popen(['perf', 'stat',  '-p', str(os.getpid())], stderr=subprocess.PIPE)

# <-- your code goes here

if os.fork() == 0:
    # child
    time.sleep(1)  # wait until parent runs `stderr.read()`
    perf.send_signal(signal.SIGINT)
    exit(0)

# parent
print("got perf stats>>{}<<".format(perf.stderr.read().decode("utf-8")))

The time.sleep(1) bit is ugly, what it does it that it will but I guess it will do the trick for 99% of the cases. It has almost no influence on the perf data, the only influence it has is on the "total runtime" (*xx seconds time elapsed)