Python Multiprocessing on List of dictionaries

问题

I have a list of dictionaries. list_of_dict = [{'name' : 'bob', 'weight': 50}, {'name' : 'ramesh', 'weight': 60}]

I want to process both the dictionaries at the same time

What should I use for this multiprocessing Pool or Process?

回答1:

I have tried with Multiprocessing Pool

from multiprocessing.pool import ThreadPool as Pool

pool_size = 5 

def worker(item1, itme2):
    try:
        print(item1.get('weight'))
        print(itme2)
    except:
        print('error with item')

pool = Pool(pool_size)
items = [{'name' : 'bob', 'weight': 50}, {'name' : 'ramesh','weight': 60}]
for item in items:
    pool.apply_async(worker, (item, 'item2'))

pool.close()
pool.join()

回答2:

I use the following because it is simple and easy to understand:

from multiprocessing.dummy import Pool as ThreadPool

def threadfunction(dict):
    weight = dict.get('weight')
    return weight

pool = ThreadPool(5)
threadresults = pool.map(threadfunction,list_of_dict)
pool.close()
pool.join()

It will handle the mapping of the the list: list_of_dict. So you just need to pass in the number of threads to the ThreadPool() function and pass the function and list to the pool.map function. Simple!

来源：https://stackoverflow.com/questions/59086617/python-multiprocessing-on-list-of-dictionaries