How to profile CPU usage of a Python script?

Question

Ideally what I want is to record the CPU usage of a Python script that is executing a deep neural net Keras model. I\'m looking for the CPU equivalent of memory_profiler, wh

Answer 1

You can run model.predict(x_test) in a subprocess and log its CPU usage simultaneously in the main process. For example,

import time
import multiprocessing as mp
import psutil
import numpy as np
from keras.models import load_model

def run_predict():
    model = load_model('1.h5')
    x_test = np.random.rand(10000, 1000)
    time.sleep(1)

    for _ in range(3):
        model.predict(x_test)
        time.sleep(0.5)

def monitor(target):
    worker_process = mp.Process(target=target)
    worker_process.start()
    p = psutil.Process(worker_process.pid)

    # log cpu usage of `worker_process` every 10 ms
    cpu_percents = []
    while worker_process.is_alive():
        cpu_percents.append(p.cpu_percent())
        time.sleep(0.01)

    worker_process.join()
    return cpu_percents

cpu_percents = monitor(target=run_predict)

The values in cpu_percents for the above script would be something like: