Multiprocessing with threading?

问题

when I trying to make my script multi-threading,

I've found out multiprocessing,

I wonder if there is a way to make multiprocessing work with threading?

• cpu 1 -> 3 threads(worker A,B,C)
• cpu 2 -> 3 threads(worker D,E,F)
• ...

Im trying to do it myself but I hit so much problems.

is there a way to make those two work together?

回答1:

You can generate a number of Processes, and then spawn Threads from inside them. Each Process can handle almost anything the standard interpreter thread can handle, so there's nothing stopping you from creating new Threads or even new Processes within each Process. As a minimal example:

def foo():
    print("Thread Executing!")

def bar():
    threads = []
    for _ in range(3): # each Process creates a number of new Threads
        thread = threading.Thread(target=foo) 
        threads.append(thread)
        thread.start()
    for thread in threads:
        thread.join()

if __name__ == "__main__": 
    processes = []
    for _ in range(3):
        p = multiprocessing.Process(target=bar) # create a new Process
        p.start()
        processes.append(p)
    for process in processes:
        process.join()

Communication between threads can be handled within each Process, and communication between the Processes can be handled at the root interpreter level using Queues or Manager objects.

回答2:

You can define a function that takes a process and make it run 3 threads and then spawn your processes to target this function, for example:

def threader(process):
    for _ in range(3):
        threading.Thread(target=yourfunc).start()

def main():
    # spawn whatever processes here to target threader

来源：https://stackoverflow.com/questions/45129894/multiprocessing-with-threading