Check if subprocess breaks and restart if True

问题

I want to do run a process in parallel on several cores. Therefore, I use the multiprocessing library in Python. However, there is a subprocess which sometimes breaks so that the whole script does not work anymore. I'd like to test whether the subprocess breaks and if this is the case, restart it.

These are the packages and data I use:

from frog import Frog, FrogOptions
import multiprocessing
textparts = ['these', 'are', 'eight', 'words', 'matching', 'the', 'eight', 'cores']

The function in which start a pool with 8 cores:

def analyse(textparts):
    p = multiprocessing.Pool(8)                     #Start a pool with 8 cores
    result = p.map(generate_pseudowords,textparts)  #collect the results, whereby generate_pseudowords 
                                                    #is the function to call and textparts the argument
                                                    #given to the function
    return(result)

Here's the function that should run in parallel on the cores:

def generate_pseudowords(textparts):
    words = []
    frog = Frog(FrogOptions(parser=False))  #initiate Frog
    words.append(frog.process(textparts))   #<--- frog.process is the critical process that sometimes break;
                                            #However, if it doesn't break, frog.process takes every item of
                                            #textparts as an argument and its output is appended in words
    return(words)

Get it running:

words = generate_pseudowords(textparts)

I cannot see an obviously reason why frog.process sometimes break, because it sometimes works with the same input and sometimes doesn't. To solve the issue, I'd like to test whether frog.process breaks and restart it, if it does.

I'll be thankful for every comment and suggestion!

来源：https://stackoverflow.com/questions/59721716/check-if-subprocess-breaks-and-restart-if-true

标签

python

multiprocessing