How to get around the pickling error of python multiprocessing without being in the top-level?

前端 未结 3 827
忘了有多久
忘了有多久 2021-02-14 12:31

I\'ve researched this question multiple times, but haven\'t found a workaround that either works in my case, or one that I understand, so please bear with me.

Basically

3条回答
  •  小鲜肉
    小鲜肉 (楼主)
    2021-02-14 12:42

    How about just taking the embedded function out?

    This seems to me the clearest solution (since you didn't give your expected output, I had to guess):

    $ cat /tmp/tmp.py
    import multiprocessing
    
    def calculate(x):
        # here is where I would take this input x (and maybe a couple more inputs)
        # and build a larger library of variables that I use further down the line
    
        pool = multiprocessing.Pool(3)
        _lst = [(x, y) for x in (x,) for y in range(3)]
        final= pool.map(domath, _lst)
        print(final)
    
    def domath(l):
        return l[0] * l[1]
    
    calculate(2)
    
    $ python /tmp/tmp.py
    [0, 2, 4]
    
    $
    

提交回复
热议问题