Python, multithreading too slow, multiprocess

Question

I\'m a multiprocessing newbie,

I know something about threading but I need to increase the speed of this calculation, hopefully with multiprocessing:

Answer 1

Just replace threading with multiprocessing and Thread with Process. Threads in Pyton are (almost) never used to gain performance because of the big bad GIL! I explained it in an another SO-post with some links to documentation and a great talk about threading in python.

But the multiprocessing module is intentionally very similar to the threading module. You can almost use it as an drop-in replacement!

The multiprocessing module doesn't AFAIK offer a functionality to enforce the use of a specific amount of cores. It relies on the OS-implementation. You could use the Pool object and limit the worker-onjects to the core-count. Or you could look for an other MPI library like pypar. Under Linux you could use a pipe under the shell to start multiple instances on different cores