Is there any simple way to benchmark python script?

Question

Usually I use shell command time. My purpose is to test if data is small, medium, large or very large set, how much time and memory usage will be.

Any t

Answer 1

The easy way to quickly test any function is to use this syntax : %timeit my_code

For instance :

%timeit a = 1

13.4 ns ± 0.781 ns per loop (mean ± std. dev. of 7 runs, 100000000 loops each)

Answer 2

snakeviz interactive viewer for cProfile

https://github.com/jiffyclub/snakeviz/

cProfile was mentioned at https://stackoverflow.com/a/1593034/895245 and snakeviz was mentioned in a comment, but I wanted to highlight it further.

It is very hard to debug program performance just by looking at cprofile / pstats output, because they can only total times per function out of the box.

However, what we really need in general is to see a nested view containing the stack traces of each call to actually find the main bottlenecks easily.

And this is exactly what snakeviz provides via its default "icicle" view.

First you have to dump the cProfile data to a binary file, and then you can snakeviz on that

pip install -u snakeviz
python -m cProfile -o results.prof myscript.py
snakeviz results.prof

This prints an URL to stdout which you can open on your browser, which contains the desired output that looks like this:

and you can then:

• hover each box to see the full path to the file that contains the function
• click on a box to make that box show up on the top as a way to zoom in

More profile oriented question: How can you profile a Python script?

Answer 3

I usually do a quick time ./script.py to see how long it takes. That does not show you the memory though, at least not as a default. You can use /usr/bin/time -v ./script.py to get a lot of information, including memory usage.

Answer 4

Memory Profiler for all your memory needs.

https://pypi.python.org/pypi/memory_profiler

Run a pip install:

pip install memory_profiler

Import the library:

import memory_profiler

Add a decorator to the item you wish to profile:

@profile
def my_func():
    a = [1] * (10 ** 6)
    b = [2] * (2 * 10 ** 7)
    del b
    return a

if __name__ == '__main__':
    my_func()

Execute the code:

python -m memory_profiler example.py

Recieve the output:

 Line #    Mem usage  Increment   Line Contents
 ==============================================
 3                           @profile
 4      5.97 MB    0.00 MB   def my_func():
 5     13.61 MB    7.64 MB       a = [1] * (10 ** 6)
 6    166.20 MB  152.59 MB       b = [2] * (2 * 10 ** 7)
 7     13.61 MB -152.59 MB       del b
 8     13.61 MB    0.00 MB       return a

Examples are from the docs, linked above.

Answer 5

Be carefull timeit is very slow, it take 12 second on my medium processor to just initialize (or maybe run the function). you can test this accepted answer

def test():
    lst = []
    for i in range(100):
        lst.append(i)

if __name__ == '__main__':
    import timeit
    print(timeit.timeit("test()", setup="from __main__ import test")) # 12 second

for simple thing I will use time instead, on my PC it return the result 0.0

import time

def test():
    lst = []
    for i in range(100):
        lst.append(i)

t1 = time.time()

test()

result = time.time() - t1
print(result) # 0.000000xxxx

Answer 6

I use a simple decorator to time the func

def st_time(func):
    """
        st decorator to calculate the total time of a func
    """

    def st_func(*args, **keyArgs):
        t1 = time.time()
        r = func(*args, **keyArgs)
        t2 = time.time()
        print "Function=%s, Time=%s" % (func.__name__, t2 - t1)
        return r

    return st_func