Multiprocessing debug techniques

Question

I\'m having trouble debugging a multi-process application (specifically using a process pool in python\'s multiprocessing module). I have an apparent deadlock and I do not know

Answer 1

In order to avoid deadlocks in the first place, learning good practices is useful, as parallel processing is indeed quite subtle. The (free) Little Book of Semaphores can be a very enjoyable read!

Answer 2

Yah, debugging deadlocks is fun. You can set the logging level to be higher -- see the Python documentation for a description of it, but really quickly:

import multiprocessing, logging
logger = multiprocessing.log_to_stderr()
logger.setLevel(multiprocessing.SUBDEBUG)

Also, add logging for anything in your code that deals with a resource or whatnot that might be in contention. Finally, shot in the dark: spawning off child processes during an import might cause a problem.