Initializing and destroying Python multiprocessing workers

前端 未结 2 1806
自闭症患者
自闭症患者 2021-01-12 03:21

I have a model which I\'m calling many times from Python. The model takes a long time to startup and shutdown, but only a short time to process the input data (which can be

相关标签:
2条回答
  • 2021-01-12 03:53

    How about

    def f(x): # worker function, runs quickly
        y = model.y
        del model
        return y * x ** 2
    
    0 讨论(0)
  • 2021-01-12 04:15

    Solution of johnthexiii would kill the model at first run of worker function. You could offer a seperate destroy function:

    import time
    
    def g(x): # destroy function
        del model
        time.sleep(1) # to ensure, this worker does not pick too many
        return None
    

    Before pool.close() you add

    pool.map_async(g, range(2), 1) # if specified to have two processes before
    

    I don’t think, this is a very “clean” solution, but it should work.

    0 讨论(0)
自定义标题
段落格式
字体
字号
代码语言
提交回复
热议问题