cprofile

I've written a small python script that uses multiprocessing (See https://stackoverflow.com/a/41875711/1878788 ). It works when I test it: $ ./forkiter.py 0 1 2 3 4 sum of x+1: 15 sum of 2*x: 20 sum of x*x: 30 But when I try to profile it with cProfile , I get the following: $ python3.6 -m cProfile -o forkiter.prof ./forkiter.py 0 1 2 3 4 Traceback (most recent call last): File "/home/bli/lib/python3.6/runpy.py", line 193, in _run_module_as_main "__main__", mod_spec) File "/home/bli/lib/python3.6/runpy.py", line 85, in _run_code exec(code, run_globals) File "/home/bli/lib/python3.6/cProfile.py

Compare a pure Python no-op function with a no-op function decorated with @numba.jit , that is: import numba @numba.njit def boring_numba(): pass def call_numba(x): for t in range(x): boring_numba() def boring_normal(): pass def call_normal(x): for t in range(x): boring_normal() If we time this with %timeit , we get the following: %timeit call_numba(int(1e7)) 792 ms ± 5.51 ms per loop (mean ± std. dev. of 7 runs, 1 loop each) %timeit call_normal(int(1e7)) 737 ms ± 2.7 ms per loop (mean ± std. dev. of 7 runs, 1 loop each) All perfectly reasonable; there's a small overhead for the numba function

I've been testing a cacheing system of my making. Its purpose is to speed up a Django web application. It stores everything in-memory. According to cProfile most of the time in my tests is spent inside QuerySet._clone() which turns out to be terribly inefficient (it's actually not that strange given the implementation). I was having high hopes for using PyPy to speed things up. I've got a 64-bit machine. However after installing all the required libraries it turns out that PyPy compiled code runs about 2.5x slower than regular Python code, and I don't know what to make out of it. The code is

I am profiling a python script main.py using cProfile with the following command: python -m cProfile -s tottime main.py The output I get is (only copy-pasted the top lines of the output): 10184337 function calls (10181667 primitive calls) in 13.597 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 1 4.674 4.674 13.598 13.598 main.py:2(<module>) 2142 2.964 0.001 4.663 0.002 load_aerdat3.py:61(getPacket) 459 2.381 0.005 2.381 0.005 {waitKey} 1667989 1.170 0.000 1.170 0.000 {numpy.core.multiarray.array} ... How can the tottime (4.674) be different

I have identified some long running pytest tests with py.test --durations=10 I would like to instrument one of those tests now with something like line_profiler or cprofile. I really want to get the profile data from the test itself as the pytest setup or tear down could well be part of what is slow. However given how line_profiler or cprofile is typically involved it isn't clear to me how to make them work with pytest. Run pytest like this: python -m cProfile -o profile $(which py.test) You can even pass in optional arguments: python -m cProfile -o profile $(which py.test) \ tests/worker/test

I would like to improve the performance of a Python script and have been using cProfile to generate a performance report: python -m cProfile -o chrX.prof ./bgchr.py ...args... I opened this chrX.prof file with Python's pstats and printed out the statistics: Python 2.7 (r27:82500, Oct 5 2010, 00:24:22) [GCC 4.1.2 20080704 (Red Hat 4.1.2-44)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import pstats >>> p = pstats.Stats('chrX.prof') >>> p.sort_stats('name') >>> p.print_stats() Sun Oct 10 00:37:30 2010 chrX.prof 8760583 function calls in 13.780 CPU seconds

I am trying to optimize some code with Cython, but cProfile is not providing enough information. To do a good job at profiling, should I create many sub-routines func2, func3,... , func40 ? Note below that i have a function func1 in mycython.pyx , but it has many for loops and internal manipulations. But cProfile does not tell me stats for those loops . 2009 function calls in 81.254 CPU seconds Ordered by: standard name ncalls tottime percall cumtime percall filename:lineno(function) 1 0.000 0.000 81.254 81.254 <string>:1(<module>) 2 0.000 0.000 0.021 0.010 blah.py:1495(len) 2000 0.000 0.000 0

As answer to my question Find the 1 based position to which two lists are the same I got the hint to use the C-library itertools to speed up things. To verify I coded the following test using cProfile: from itertools import takewhile, izip def match_iter(self, other): return sum(1 for x in takewhile(lambda x: x[0] == x[1], izip(self, other))) def match_loop(self, other): element = -1 for element in range(min(len(self), len(other))): if self[element] != other[element]: element -= 1 break return element +1 def test(): a = [0, 1, 2, 3, 4] b = [0, 1, 2, 3, 4, 0] print("match_loop a=%s, b=%s,

Hey. I'm working on an App Engine app that involves queries to the Google Maps API for geocoding. Google Maps doesn't like too much requests so I put a 1 second delay between each request with time.sleep(1) . I noticed that my quotas are running low in my GAE dashboard and decided to run a short test: import cProfile import time def foo(): time.sleep(3) cProfile.run('foo()') Which gave me the following output: 4 function calls in 3.003 CPU seconds Ordered by: standard name ncalls tottime percall cumtime percall filename:lineno(function) 1 0.000 0.000 3.003 3.003 <stdin>:1(foo) 1 0.000 0.000 3