Purpose of multiprocessing.Pool.apply and multiprocessing.Pool.apply_async

Question

See example and execution result below:

#!/usr/bin/env python3.4
from multiprocessing import Pool
import time
import os

def initializer():
    print(\"In initia         


        

Answer 1

This line in your code:

Pool(processes=4, initializer=initializer) as pool:  # start 4 worker processes

doesn't start 4 worker processes. It just create a pool of them than can support running that many of them at a time running concurrently. It's methods like apply() that actually start separate processes running.

The difference is that apply() and apply_async() is that the former blocks until a result is ready but the latter returns a "result" object right away. This doesn't make much difference unless you want to submit more than one task to the Pool at a time (which of course is the whole point of using the multiprocessing module).

Here's are some modifications to your code showing how to actually do some concurrent processing with the Pool:

from multiprocessing import Pool
import time
import os

def initializer():
    print("In initializer pid is {} ppid is {}".format(os.getpid(),os.getppid()))

def f(x):
    print("In f pid is {} ppid is {}".format(os.getpid(),os.getppid()))
    return x*x

if __name__ == '__main__':
    print("In main pid is {} ppid is {}".format(os.getpid(), os.getppid()))
    with Pool(processes=4, initializer=initializer) as pool:  # Create 4 worker Pool.
#        result = pool.apply(f, (10,)) # evaluate "f(10)" in a single process
#        print(result)
        # Start multiple tasks.
        tasks = [pool.apply_async(f, (val,)) for val in range(10, 20)]
        pool.close()  # No more tasks.
        pool.join()  # Wait for all tasks to finish.
        results = [result.get() for result in tasks]  # Get the result of each.
        print(results)

map_sync() would be better suited for processing something like this (a sequence of values) as it will handle some of the details shown in the above code automatically.