In Python, why is a module implemented in C faster than a pure Python module, and how do I write one?

Question

The python documentation states, that the reason cPickle is faster than Pickle is, that the former is implemented in C. What does that mean exactly?

I am making a module

Answer 1

Besides Pyrex/Cython, already mentioned, you have other alternatives:

Shed Skin: Translates (a restricted subset of) Python to C++. Can automatically generate an extension for you. You'd create an extension doing this (assuming Linux):

wget http://shedskin.googlecode.com/files/shedskin-0.7.tgz
tar -xzf shedskin-0.7.tgz
# On your code folder:
PYTHONPATH=/path/to/shedskin-0.7 python shedskin -e yourmodule.py
# The above generates a Makefile and a yourmodule.h/.cpp pair
make
# Now you can "import yourmodule" from Python and check it's from the .so by "print yourmodule.__file__

PyPy: A faster Python, with a JIT compiler. You could simply run your code on it instead of CPython. Only supports Python 2.5 now, 2.7 support soon. Can give huge speedups on math-heavy code. To install and run it (assuming Linux 32-bit):

wget http://pypy.org/download/pypy-1.4.1-linux.tar.bz2
tar -xjf pypy-1.4.1-linux.tar.bz2
sudo ln -s /path/to/pypy-1.4.1-linux/bin/pypy /usr/local/bin
# Then, instead of "python yourprogram.py" you'll just run "pypy yourprogram.py"

Weave: Allows you to write C inline, the compiles it.

Edit: If you want us to run these tools for you and benchmark, just post your code ;)