问题
I'm using python 2.7. Here's some of my code:
from pathos.multiprocessing import ProcessingPool as Pool
class pair_scraper(object):
#code and various methods ...
def parallelized_processing(self):
'''
Parallelize length extraction of
relevant files.
'''
pool = Pool(self.pool_size)
pool.map(self.length_processer, self.zip_files)
pool.close()
pool.join()
However when I run this, I get a blank AssertionError
:
/...file.py in parallelized_processing(self)
69 '''
70 pool = Pool(self.pool_size)
---> 71 pool.map(self.length_processer, self.zip_files)
72 # pool.close()
73 # pool.join()
/.../anaconda/lib/python2.7/site-packages/pathos/multiprocessing.pyc in map(self, f, *args, **kwds)
134 AbstractWorkerPool._AbstractWorkerPool__map(self, f, *args, **kwds)
135 _pool = self._serve()
--> 136 return _pool.map(star(f), zip(*args)) # chunksize
137 map.__doc__ = AbstractWorkerPool.map.__doc__
138 def imap(self, f, *args, **kwds):
/.../anaconda/lib/python2.7/site-packages/multiprocess/pool.pyc in map(self, func, iterable, chunksize)
248 Equivalent of `map()` builtin
249 '''
--> 250 assert self._state == RUN
251 return self.map_async(func, iterable, chunksize).get()
252
AssertionError:
This error seems to resemble the one described in this question, although the fix there was more obvious. I ran the above code removing pool.close()
and pool.join()
but got the same results.
Thoughts?
来源:https://stackoverflow.com/questions/40734063/assertionerror-using-pathos-multiprocessing