Speeding Up Python

Question

This is really two questions, but they are so similar, and to keep it simple, I figured I\'d just roll them together:

• Firstly: Given an est

Answer 1

A couple of ways to speed up Python code were introduced after this question was asked:

• Pypy has a JIT-compiler, which makes it a lot faster for CPU-bound code.
• Pypy is written in Rpython, a subset of Python that compiles to native code, leveraging the LLVM tool-chain.

Answer 2

This won't necessarily speed up any of your code, but is critical knowledge when programming in Python if you want to avoid slowing your code down. The "Global Interpreter Lock" (GIL), has the potential to drastically reduce the speed of your multi-threaded program if its behavior is not understood (yes, this bit me ... I had a nice 4 processor machine that wouldn't use more than 1.2 processors at a time). There's an introductory article with some links to get you started at SmoothSpan.

Answer 3

Cython and pyrex can be used to generate c code using a python-like syntax. Psyco is also fantastic for appropriate projects (sometimes you'll not notice much speed boost, sometimes it'll be as much as 50x as fast). I still reckon the best way is to profile your code (cProfile, etc.) and then just code the bottlenecks as c functions for python.

Answer 4

I hope you've read: http://wiki.python.org/moin/PythonSpeed/PerformanceTips

Resuming what's already there are usualy 3 principles:

• write code that gets transformed in better bytecode, like, use locals, avoid unnecessary lookups/calls, use idiomatic constructs (if there's natural syntax for what you want, use it - usually faster. eg: don't do: "for key in some_dict.keys()", do "for key in some_dict")
• whatever is written in C is considerably faster, abuse whatever C functions/modules you have available
• when in doubt, import timeit, profile

Answer 5

For an established project I feel the main performance gain will be from making use of python internal lib as much as possible.

Some tips are here: http://blog.hackerearth.com/faster-python-code

Answer 6

Just a note on using psyco: In some cases it can actually produce slower run-times. Especially when trying to use psyco with code that was written in C. I can't remember the the article I read this, but the map() and reduce() functions were mentioned specifically. Luckily you can tell psyco not to handle specified functions and/or modules.