Freeze when using tkinter + pyhook. Two event loops and multithreading

Question

I am writing a tool in python 2.7 registering the amount of times the user pressed a keyboard or mouse button. The amount of clicks will be displayed in a small black box in the top left of the screen. The program registers clicks even when another application is the active one.

It works fine except when I move the mouse over the box. The mouse then freezes for a few seconds after which the program works again. If I then move the mouse over the box a second time, the mouse freezes again, but this time the program crashes.

I have tried commenting out pumpMessages() and then the program works. The problem looks a lot like this question pyhook+tkinter=crash, but no solution was given there.

Other answers has shown that there is a bug with the dll files when using wx and pyhook together in python 2.6. I don't know if that is relevant here.

My own thoughts is that it might have something to do with the two event loops running parallel. I have read that tkinter isn't thread safe, but I can't see how I can make this program run in a single thread since I need to have both pumpmessages() and mainlooop() running.

To sum it up: Why does my program freeze on mouse over?

import pythoncom, pyHook, time, ctypes, sys
from Tkinter import *
from threading import Thread

print 'Welcome to APMtool. To exit the program press delete'

## Creating input hooks

#the function called when a MouseAllButtonsUp event is called
def OnMouseUpEvent(event): 
    global clicks
    clicks+=1
    updateCounter()
    return True

#the function called when a KeyUp event is called
def OnKeyUpEvent(event): 
    global clicks
    clicks+=1
    updateCounter()
    if (event.KeyID == 46):
        killProgram()
    return True


hm = pyHook.HookManager()# create a hook manager

# watch for mouseUp and keyUp events
hm.SubscribeMouseAllButtonsUp(OnMouseUpEvent)
hm.SubscribeKeyUp(OnKeyUpEvent)

clicks = 0

hm.HookMouse()# set the hook
hm.HookKeyboard()

## Creating the window
root = Tk()
label = Label(root,text='something',background='black',foreground='grey')
label.pack(pady=0) #no space around the label
root.wm_attributes("-topmost", 1) #alway the top window
root.overrideredirect(1) #removes the 'Windows 7' box around the label

## starting a new thread to run pumMessages() and mainloop() simultaniusly
def startRootThread():
    root.mainloop()

def updateCounter():
    label.configure(text=clicks)

def killProgram():
    ctypes.windll.user32.PostQuitMessage(0) # stops pumpMessages
    root.destroy() #stops the root widget
    rootThread.join()
    print 'rootThread stopped'



rootThread = Thread(target=startRootThread)
rootThread.start()

pythoncom.PumpMessages() #pump messages is a infinite loop waiting for events

print 'PumpMessages stopped'

Answer 1

I've solved this problem with multiprocessing:

1. the main process handles the GUI (MainThread) and a thread that consumes messages from the second process

2. a child process hooks all mouse/keyboard events and pushes them to the main process (via a Queue object)

Answer 2

From the information that Tkinter needs to run in the main thread and not be called outside this thred, I found a solution:

My problem was that both PumpMessages and mainLoop needed to run in the main thread. In order to both receive inputs and show a Tkinter label with the amount of clicks I need to switch between running pumpMessages and briefly running mainLoop to update the display.

To make mainLoop() quit itself I used:

after(100,root.quit()) #root is the name of the Tk()
mainLoop()

so after 100 milliseconds root calls it's quit method and breaks out of its own main loop

To break out of pumpMessages I first found the pointer to the main thread:

mainThreadId = win32api.GetCurrentThreadId()

I then used a new thread that sends the WM_QUIT to the main thread (note PostQuitMessage(0) only works if it is called in the main thread):

win32api.PostThreadMessage(mainThreadId, win32con.WM_QUIT, 0, 0)

It was then possible to create a while loop which changed between pumpMessages and mainLoop, updating the labeltext in between. After the two event loops aren't running simultaneously anymore, I have had no problems:

def startTimerThread():
    while True:
        win32api.PostThreadMessage(mainThreadId, win32con.WM_QUIT, 0, 0)
        time.sleep(1)

mainThreadId = win32api.GetCurrentThreadId()
timerThread = Thread(target=startTimerThread)
timerThread.start()

while programRunning:
    label.configure(text=clicks)
    root.after(100,root.quit)
    root.mainloop()
    pythoncom.PumpMessages()

Thank you to Bryan Oakley for information about Tkinter and Boaz Yaniv for providing the information needed to stop pumpMessages() from a subthread

Answer 3

Tkinter isn't designed to be run from any thread other than the main one. It might help to put the GUI in the main thread and put the call to PumpMessages in a separate thread. Though you have to be careful and not call any Tkinter functions from the other thread (except perhaps event_generate).