saving cProfile results to readable external file

后端 未结 5 1755
清酒与你
清酒与你 2021-02-19 01:16

I am using cProfile try to profile my codes:

pr = cProfile.Profile()
pr.enable()
my_func()   # the code I want to profile
pr.disable()
pr.print_stat         


        
相关标签:
5条回答
  • 2021-02-19 01:56

    Updated. You can get output of profiler using io.StringIO() and save it into file. Here is an example:

    import cProfile
    import pstats
    import io
    
    
    def my_func():
        result = []
        for i in range(10000):
            result.append(i)
    
        return result
    
    pr = cProfile.Profile()
    pr.enable()
    
    my_result = my_func()
    
    pr.disable()
    s = io.StringIO()
    ps = pstats.Stats(pr, stream=s).sort_stats('tottime')
    ps.print_stats()
    
    with open('test.txt', 'w+') as f:
        f.write(s.getvalue())
    

    Run our script and open test.txt. You will see readable result:

       10002 function calls in 0.003 seconds
    
       Ordered by: internal time
    
       ncalls  tottime  percall  cumtime  percall filename:lineno(function)
            1    0.002    0.002    0.003    0.003 /path_to_script.py:26(my_func)
        10000    0.001    0.000    0.001    0.000 {method 'append' of 'list' objects}
            1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
    

    Also I can recommend to use dump_stats + pstats.Stats. Here is an example how to use it. Structure of files:

    # test_ex.py - just a small web app
    import cProfile
    import json
    from functools import wraps
    from flask import Flask
    
    from example.mod1 import func1
    from example.mod2 import func2
    
    app = Flask(__name__)
    
    # profiling decorator
    def profiling():
    
        def _profiling(f):
            @wraps(f)
            def __profiling(*rgs, **kwargs):
                pr = cProfile.Profile()
                pr.enable()
    
                result = f(*rgs, **kwargs)
    
                pr.disable()
                # save stats into file
                pr.dump_stats('profile_dump')
    
                return result
            return __profiling
        return _profiling
    
    # demonstration route with profiler
    @app.route('/test')
    @profiling()
    def test():
        counter = func1()
        dict_data = func2()
        result = dict()
    
        for key, val in dict_data.items():
            result[key] = val + counter
    
        return json.dumps(result)
    
    
    if __name__ == '__main__':
        app.run(debug=True, port=8083)
    

    example package - let's imagine that this is some kind of application logic.

    # example.mod1
    def func1():
        counter = 0
    
        for i in range(100000):
            counter += i
    
        return counter
    
    # example.mod2
    def func2():
        res = dict()
    
        for i in range(300000):
            res['key_' + str(i)] = i
    
        return res
    

    Now let's run server(python3 test_ex.py) and open http://localhost:8083/test. After a few seconds you will see long json. After that you will see profile_dump file in project folder. Now run python live interpreter in project folder and print our dump using pstats:

    import pstats
    p = pstats.Stats('profile_dump')
    # skip strip_dirs() if you want to see full path's
    p.strip_dirs().print_stats()
    

    Also you can easy sorting results:

    p.strip_dirs().sort_stats('tottime').print_stats()
    p.strip_dirs().sort_stats('cumulative').print_stats()
    p.strip_dirs().sort_stats().print_stats('mod1')
    

    Hope this helps.

    0 讨论(0)
  • 2021-02-19 02:00

    You can run the profiler saving the output into a file, as you do:

    import cProfile
    
    cProfile.run('my_func()', 'profile_results')
    

    And then format that result using the class pstats.Stats (https://docs.python.org/3/library/profile.html#the-stats-class):

    import pstats
    
    file = open('formatted_profile.txt', 'w')
    profile = pstats.Stats('.\profile_results', stream=file)
    profile.sort_stats('cumulative') # Sorts the result according to the supplied criteria
    profile.print_stats(15) # Prints the first 15 lines of the sorted report
    file.close()
    
    0 讨论(0)
  • 2021-02-19 02:01

    Expanding upon the previous answer, you can dump everything out to a .csv file to sort and play around with in your favorite spreadsheet application.

    import pstats,StringIO
    
    # print stats to a string
    result=StringIO.StringIO()
    pstats.Stats(filename,stream=result).print_stats()
    result=result.getvalue()
    
    # chop the string into a csv-like buffer
    result='ncalls'+result.split('ncalls')[-1]
    result='\n'.join([','.join(line.rstrip().split(None,6)) for line in result.split('\n')])
    
    # save it to disk
    f=open(filename.rsplit('.')[0]+'.csv','w')
    f.write(result)
    f.close()
    
    0 讨论(0)
  • 2021-02-19 02:02

    you can use dump_stats. In Python 3.8:

    with cProfile.Profile() as pr:
        my_func()  
    
    pr.dump_stats('/path/to/filename.prof')
    
    0 讨论(0)
  • 2021-02-19 02:05

    You don't really need StringIO, as a file qualifies as a stream.

    import pstats
    
    with open("profilingStatsAsText.txt", "w") as f:
        ps = pstats.Stats("profilingResults.cprof", stream=f)
        ps.sort_stats('cumulative')
        ps.print_stats()
    
    0 讨论(0)
提交回复
热议问题