python-multiprocessing

问题 This question concerns multiprocessing in python. I want to execute some code when I terminate the process, to be more specific just before it will be terminated. I'm looking for a solution which works as atexit.register for the python program. I have a method worker which looks: def worker(): while True: print('work') time.sleep(2) return I run it by: proc = multiprocessing.Process(target=worker, args=()) proc.start() My goal is to execute some extra code just before terminating it, which I

问题 I have a dictionary that looks like this: sampleData = {'x1': [1,2,3], 'x2': [4,5,6], 'x3': [7,8,9]} I need to do some calculation for each key and value pair by passing data to a blackBoxFunction . This function takes time to do the processing. The final output is stored in a separate dictionary finalValue = {} . This is the code for doing it sequentially: for key in sampleData.keys(): finalValue[key] = [] for i in range(0,len(sampleData[key])): for j in range(i,len(sampleData[key])): if(i!

问题 When you supply a large-enough object into multiprocessing.Queue , the program seems to hang at weird places. Consider this minimal example: import multiprocessing def dump_dict(queue, size): queue.put({x: x for x in range(size)}) print("Dump finished") if __name__ == '__main__': SIZE = int(1e5) queue = multiprocessing.Queue() process = multiprocessing.Process(target=dump_dict, args=(queue, SIZE)) print("Starting...") process.start() print("Joining...") process.join() print("Done") print(len

问题 The idea im trying to run is this: RUN 3 Processes doing calculation ONCE one of the 3 processes finishes the task KILL others imediatly and continue with the main task, i can't let it run any second longer The things i've tried was: Putting the global variable through multiprocessing.manager, but that still lets processes finish their loops. Raising an exception OS: Windows PYTHON: 2.7 def f(name): Doing = True try: while Doing: print 'DOING',name somecodethatmarksDoingAsFalse() except

问题 When using map() from multiprocessing.Pool() on a list of instances from a numpy.ndarray -subclass, the new attributes of the own class are dropped. The following minimal example based on the numpy docs subclassing example reproduces the problem: from multiprocessing import Pool import numpy as np class MyArray(np.ndarray): def __new__(cls, input_array, info=None): obj = np.asarray(input_array).view(cls) obj.info = info return obj def __array_finalize__(self, obj): if obj is None: return self

问题 I need to do some stuffs in multiprocess with Python 3.6. Namely, I have to update a dict adding lists of objects. Since these objects are unpickable I need to use dill instead of pickle and multiprocess from pathos instead of multiprocessing , but this should not be the problem. Adding a list to the dictionary needs to reserialize the list before of adding to the dictionary. This slow down everything and it takes the same time as without multiprocessing. Could you suggest me a workaround?

问题 Given a list() of running multiprocessing.Process -instances, how can I join on all of them and return as soon as one exits without a Process.join -timeout and looping? Example from multiprocessing import Process from random import randint from time import sleep def run(): sleep(randint(0,5)) running = [ Process(target=run) for i in range(10) ] for p in running: p.start() How can I block until at least one Process in p exits? What I don't want to do is: exit = False while not exit: for p in

问题 import time from multiprocessing import Process start = time.perf_counter() def sleep(): print('Sleeping 1 second(s)...') time.sleep(1) return 'Done Sleeping...' p1 = Process(target = sleep) p2 = Process(target = sleep) p1.start() p2.start() p1.join() p2.join() finish = time.perf_counter() print(f'Finished in {round(finish-start, 2)} second(s)') output: Finished in 0.17 second(s) I tried to use multiprocessing, but when I run the code it`s over in 0.17~ seconds and not 1 as it supposed to be,

问题 import time from multiprocessing import Process start = time.perf_counter() def sleep(): print('Sleeping 1 second(s)...') time.sleep(1) return 'Done Sleeping...' p1 = Process(target = sleep) p2 = Process(target = sleep) p1.start() p2.start() p1.join() p2.join() finish = time.perf_counter() print(f'Finished in {round(finish-start, 2)} second(s)') output: Finished in 0.17 second(s) I tried to use multiprocessing, but when I run the code it`s over in 0.17~ seconds and not 1 as it supposed to be,

问题 import time from multiprocessing import Process start = time.perf_counter() def sleep(): print('Sleeping 1 second(s)...') time.sleep(1) return 'Done Sleeping...' p1 = Process(target = sleep) p2 = Process(target = sleep) p1.start() p2.start() p1.join() p2.join() finish = time.perf_counter() print(f'Finished in {round(finish-start, 2)} second(s)') output: Finished in 0.17 second(s) I tried to use multiprocessing, but when I run the code it`s over in 0.17~ seconds and not 1 as it supposed to be,