问题
I'm using DEAP library to implement genetic programming and I have used eaMuCommaLambda algorithm for this purpose. In order to run the program in parallel, I followed the instructions in the DEAP document and added the two following lines of code in the if __name__ == "__main__" section.
import multiprocessing
pool = multiprocessing.Pool()
toolbox.register("map", pool.map)
pop, log = algorithms.eaMuCommaLambda(pop, toolbox, MU, LAMBDA, cxpb, mutpb, gen, halloffame=hof, stats=mstats, verbose=True)
In the source code of the eaMuCommaLambda algorithm, evaluation operation is mapped as follows:
fitnesses = toolbox.map(toolbox.evaluate, invalid_ind)
Thus, by replacing the default map with the pool.map, it is expected to have the evaluation operations in parallel. The program runs with no error, but it is not doing anything. At first I can see several processes start in the task manager, but soon their CPU usage drops to zero and the program keeps running while it seems nothing is actually calculated. The evaluation function is not carried out at all. The code works fine without multiprocessing, but I'm not sure why multiprocessing is not working properly. I'll be grateful if anyone could suggest what might be the reason.
回答1:
After to your evaluation algorithm, please add the below code and try,
pool.close()
This will make sure to close all the pools you have started.
Hope this speeds up your process.
回答2:
I was using PyCharm to run the code in Windows, and it seems the multiprocessing.Pool cannot be used inside interactive interpreter. More information can be found in the following link: https://intellij-support.jetbrains.com/hc/en-us/community/posts/115000384464-Problem-using-multiprocess-with-IPython
The problem was solved by running the code in cmd.
来源：https://stackoverflow.com/questions/59116521/using-multiprocessing-in-deap-for-genetic-programming