问题
I am creating multiple processes which are running crawlers separately. I want to ensure that if there is some exception in crawler process, I am able to catch it in Parent Process. Here is the process creation code:
try:
caching_process = Process(target=run_crawler_process, args=(Config.CRAWLER_NAME, locations,
city_payloads_map, cycle_count))
caching_process.start()
except Exception as processException:
raise processException
回答1:
You cannot do that with Process
objects. The multiprocessing.Pool or the concurrent.futures.ProcessPoolExecutor allow to do that.
pool = multiprocessing.Pool()
task = pool.apply_async(run_crawler_process, (Config.CRAWLER_NAME, locations,c ity_payloads_map, cycle_count))
try:
task.get()
except Exception as error:
print("Error while processing task: %s" % error)
来源:https://stackoverflow.com/questions/60614531/catch-child-process-exception-in-parent-process