问题
I'm trying to use multiprocessing and matplotlib together.
I'm creating a standard Pool, adding work with apply_async and updating the GUI with apply_async's callback function, which runs on the Pool's parent process (I verified this with os.getpid()). Example :
from pylab import *
from numpy import *
from numpy.random import random
from multiprocessing import Pool
# Output image
global out_all
out_all = zeros((256, 256))
# Only does something to in_image, doesn't access anything else
def do_work(in_image):
for x in xrange(100000):
out_image = in_image[::-1, ::-1]
return out_image
# Update the output image and display if needed
def do_update(out_image):
global out_all
print ("Updating")
out_all += out_image
clf()
imshow(out_all)
show()
# Input images (close enough to what I do as well)
work = [random((256, 256)) for f in range(20)]
# Don't block when showing something
ion()
# Do the work
print "Starting pool"
pool = Pool()
for o in work:
pool.apply_async(do_work, [o], callback=do_update).get()
pool.close()
pool.join()
print "Stopping pool"
# Block
ioff()
show()
print "Done"
The processing itself works fine, the processes are really destroyed on pool.join(), but Matplotlib (and TK, I guess) complain as soon as I try to do something afterwards, even only exiting the program :
Traceback (most recent call last):
File "test_thread.py", line 27, in <module>
show()
File "/usr/lib/pymodules/python2.7/matplotlib/pyplot.py", line 139, in show
_show(*args, **kw)
File "/usr/lib/pymodules/python2.7/matplotlib/backend_bases.py", line 83, in __call__
manager.show()
File "/usr/lib/pymodules/python2.7/matplotlib/backends/backend_tkagg.py", line 444, in show
self.canvas.draw_idle()
File "/usr/lib/pymodules/python2.7/matplotlib/backends/backend_tkagg.py", line 258, in draw_idle
self._idle_callback = self._tkcanvas.after_idle(idle_draw)
File "/usr/lib/python2.7/lib-tk/Tkinter.py", line 512, in after_idle
return self.after('idle', func, *args)
File "/usr/lib/python2.7/lib-tk/Tkinter.py", line 504, in after
name = self._register(callit)
File "/usr/lib/python2.7/lib-tk/Tkinter.py", line 1101, in _register
self.tk.createcommand(name, f)
RuntimeError: main thread is not in main loop
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
File "/usr/lib/python2.7/atexit.py", line 24, in _run_exitfuncs
func(*targs, **kargs)
File "/usr/lib/pymodules/python2.7/matplotlib/_pylab_helpers.py", line 82, in destroy_all
manager.destroy()
File "/usr/lib/pymodules/python2.7/matplotlib/backends/backend_tkagg.py", line 452, in destroy
self.canvas._tkcanvas.after_cancel(self.canvas._idle_callback)
File "/usr/lib/python2.7/lib-tk/Tkinter.py", line 519, in after_cancel
data = self.tk.call('after', 'info', id)
RuntimeError: main thread is not in main loop
Error in sys.exitfunc:
Traceback (most recent call last):
File "/usr/lib/python2.7/atexit.py", line 24, in _run_exitfuncs
func(*targs, **kargs)
File "/usr/lib/pymodules/python2.7/matplotlib/_pylab_helpers.py", line 82, in destroy_all
manager.destroy()
File "/usr/lib/pymodules/python2.7/matplotlib/backends/backend_tkagg.py", line 452, in destroy
self.canvas._tkcanvas.after_cancel(self.canvas._idle_callback)
File "/usr/lib/python2.7/lib-tk/Tkinter.py", line 519, in after_cancel
data = self.tk.call('after', 'info', id)
RuntimeError: main thread is not in main loop
My first thought was that the TK context was duplicated on each fork(), which somehow interferred with the TK loop in the main process, but I'm not doing anything TK-related in my workers. Any ideas?
回答1:
The error messages reference Tkinter. So it looks like you are using the TkAgg backend. The code that follows is TkAgg/Tkinter-specific. In particular the call
win.after(100, animate)
makes use of the Tkinter-specific after method. There is an analogous call for GtkAgg/PyGtk, and similarly for other backends. But I just want to stress that what follows is TkAgg/Tkinter-specific.
Tkinter is intended to be run in a single thread. That is, all Tkinter GUI calls should originate from a single thread (usually, be not necessarily, the main thread).
The Pool's apply_async callback method runs in a separate (_handle_results) thread in the main process. Since imshow() is called from the Pool's _handle_results thread and show() is called in the main thread, Tkinter complains
RuntimeError: main thread is not in main loop
I don't see a way to use the apply_async callback in this situation.
Instead, what we can do is arrange for do_work to put out_image in a multiprocessing.Queue() (which I call out_queue in the code below). We'll then have the main process's main thread poll this Queue for items and display them as they come out of the Queue. This polling is done in the animate function, below.
plt.ion() is meant for interactive sessions only. Although it is sometimes possible to write little scripts which sort-of seem to work with plt.ion(), you'll get better results and cleaner GUIs if you resist using plt.ion() in scripts and instead write code that respects the GUI framework's event loop.
Although it is probably possible to fix your script and use plt.ion(), since this is not the recommended way of writing matplotlib scripts, let's see if we can avoid doing that.
plt.show() tells Tkinter to run its event loop. Notice that once this call is made, the GUI window is drawn, you can click buttons, zoom in and out, etc.
Somehow we need to inject a function into this event loop, to be run periodically by the event loop, and cooperatively with all the other GUI events that might be occurring. We want this function to check if any of our worker subprocesses have output for us, and if so, to update the imshow image.
With TkAgg/Tkinter, the way to inject such a function is
win = fig.canvas.manager.window
win.after(100, animate)
This will tell Tkinter to run the function animate (once) after (about) 100ms have elapsed. Since we want the function animate to run periodically, we just stick another
win.after(100, animate)
call at the end of animate.
import matplotlib as mpl
mpl.use('TkAgg')
import matplotlib.pyplot as plt
import numpy as np
import multiprocessing as mp
import logging
import Queue
logger = mp.log_to_stderr(logging.INFO)
# Only does something to in_image, doesn't access anything else
def do_work(in_image):
logger.info('Processing in_image')
for x in xrange(100000):
out_image = in_image[::-1, ::-1]
out_queue.put(out_image)
# Update the output image and display if needed
out_all = np.zeros((256, 256))
def pool_initializer(out_queue_):
# Setup out_queue as a global variable *in the worker subprocesses*
global out_queue
out_queue = out_queue_
def animate():
global out_all
try:
out_image = out_queue.get_nowait()
except Queue.Empty:
pass
else:
logger.info("Updating")
out_all += out_image
im.set_data(out_all)
fig.canvas.draw() # redraw the canvas
win.after(100, animate)
if __name__ == '__main__':
out_queue = mp.Queue()
logger.info("Starting pool")
pool = mp.Pool(initializer=pool_initializer, initargs=(out_queue, ))
work = [np.random.random((256, 256)) for f in range(20)]
for o in work:
pool.apply_async(do_work, [o])
pool.close()
fig, ax = plt.subplots()
win = fig.canvas.manager.window
# Output image
im = plt.imshow(out_all, vmin=0, vmax=1)
# Register a function to be run once
win.after(100, animate)
plt.show()
logger.info("Done")
来源：https://stackoverflow.com/questions/16016102/matplotlib-and-multiprocessing-runtimeerror