How can I tell where my python script is hanging?

Question

So I\'m debugging my python program and have encountered a bug that makes the program hang, as if in an infinite loop. Now, I had a problem with an infinite loop before, but

Answer 1

i = 0
for t in threading.enumerate():
    if i != 0:# and t.getName() != 'Thread-1':
        print t.getName()
        t._Thread__stop()
    i += 1

Once you know the names of the threads; start re-executing your script and filter them down, not stopping them from being aborted. i=0 conditional prevents the main thread from being aborted.

I suggest going through and naming all your threads; such as: Thread(target=self.log_sequence_status, name='log status')

This code should be placed at the end of the main program that starts up the run-away process

Answer 2

Wow! 5 answers already and nobody has suggested the most obvious and simple:

1. Try to find a reproducible test case that causes the hanging behavior.
2. Add logging to your code. This can be as basic as print "**010", print "**020", etc. peppered through major areas.
3. Run code. See where it hangs. Can't understand why? Add more logging. (I.e. if between **020 and **030, go and add **023, **025, **027, etc.)
4. Goto 3.

Answer 3

If your program is a bit too complex to simply trace all the functions, you can try running it and manually attaching a tracer program like lptrace to it. It works a bit like strace– it prints every function call your program makes. Here's how to call it:

python lptrace -p $STUCK_PROGRAM_PID

Note that lptrace requires gdb to run.

Answer 4

Haven't used it myself but I've heard that the Eric IDE is good and has a good debugger. That's also the only IDE I know of that has a debugger for Python

Answer 5

It's easier to prevent these hang-ups than it is to debug them.

First: for loops are very, very hard to get stuck in a situation where the loop won't terminate. Very hard.

Second: while loops are relatively easy to get stuck in a loop.

The first pass is to check every while loop to see if it must be a while loop. Often you can replace while constructs with for, and you'll correct your problem by rethinking your loop.

If you cannot replace a while loop with for, then you simply have to prove that the expression in the while statement must change every time through the loop. This isn't that hard to prove.

1. Look at all the condition in the loop. Call this T.

2. Look at all the logic branches in the body of the loop. Is there any way to get through the loop without making a change to the condition, T?

   • Yes? That's your bug. That logic path is wrong.

   • No? Excellent, that loop must terminate.

Answer 6

I wrote a module that prints out threads that hang longer that 10 seconds at one place. hanging_threads.py

Run:

python -m pip install hanging_threads

Add this to your code:

from hanging_threads import start_monitoring
start_monitoring(seconds_frozen=10, test_interval=100)

Here is an example output:

--------------------    Thread 5588     --------------------
  File "C:\python33\lib\threading.py", line 844, in _exitfunc
        t.join()
  File "C:\python33\lib\threading.py", line 743, in join
        self._block.wait()
  File "C:\python33\lib\threading.py", line 184, in wait
        waiter.acquire()

This occurs at the exit of the main thread when you forget to set another thread as daemon.