Catch child process exception in parent process

a 夏天 提交于 2021-01-28 08:40:59

问题


I am creating multiple processes which are running crawlers separately. I want to ensure that if there is some exception in crawler process, I am able to catch it in Parent Process. Here is the process creation code:

try:
    caching_process = Process(target=run_crawler_process, args=(Config.CRAWLER_NAME, locations,
                                                               city_payloads_map, cycle_count))
    caching_process.start()
except Exception as processException:
    raise processException

回答1:


You cannot do that with Process objects. The multiprocessing.Pool or the concurrent.futures.ProcessPoolExecutor allow to do that.

pool = multiprocessing.Pool()
task = pool.apply_async(run_crawler_process, (Config.CRAWLER_NAME, locations,c ity_payloads_map, cycle_count))

try:
    task.get()
except Exception as error:  
    print("Error while processing task: %s" % error)


来源:https://stackoverflow.com/questions/60614531/catch-child-process-exception-in-parent-process

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!