How to have a thread use the next object not already used?

问题

It's possible I might be trying to use threading for something I shouldn't be, if so please let me know.

I made a smaller example of my code so it's simpler to understand what I'd like.

I'm having users input IP addresses, then i'm pinging the IP addresses they entered. I'd like to ping x number of IP addresses at a time, in this example its 4.

The issue is that each thread pings all 4 IP addresses. I'd like 1 thread to ping IP 1, then another to ping IP 2, then if thread 1 is fished with IP 1 it moves to the next available IP (so if thread 2 picked up IP 3, then thread 1 would pick up IP4).

This way I can ping all IP addresses in ~2 seconds, rather than ~4 seconds.

Let me know if you need more details.

import threading
#prints out "pinging IP" for each IP in ip_list
def _ping(ip_list):
    for objects in ip_list:
        print ("Pinging ip: " + objects)

ip_list = []

#ask user for IP address, appends to ip_list
for x in range (0,4):
    ip = str(input("please enter ip: "))
    ip_list.append(ip)

#calls function _ping with threading.
if __name__ == '__main__':
    numOfThreads = 2
    threadList = []
    for i in range (numOfThreads):
        t = threading.Thread(target=_ping, args=(ip_list[0:4],))
        t.start()
        threadList.append(t)

Output is:

please enter ip: 1
please enter ip: 2
please enter ip: 3
please enter ip: 4
Pinging ip: 1
Pinging ip: 2
Pinging ip: 3
Pinging ip: 4
Pinging ip: 1
Pinging ip: 2
Pinging ip: 3
Pinging ip: 4

回答1:

A queue is a good data structure to help solve this. Queues are first-in, first-out (FIFO) lists, meaning that items are removed from it in the order in which they are put into it. The queue module in Python's standard library implements a thread-safe queue, exactly what you need.

You can have the main thread place all the IPs into the queue, then each thread will dequeue from it. This will remove the item, so that threads only get an IP if it's not already been processed. This will require only a slight modification to your existing code.

The most important change is that you need some way of shutting the threads down when there are no more elements to process. The most common way to do this is to add a "sentinel", some object which notifies the thread to shutdown. None is a common choice, as is an empty string.

import threading
import queue

def _ping(ip_queue):
    while True:
        ip = ip_queue.get()
        if ip is None: # Loop until sentinel encountered
            return
        print("Pinging ip: " + ip)

# Construct a queue, rather than a simple list
ip_queue = queue.Queue()

# Ask user for IP address, place onto queue
for x in range (4):
    ip = input("please enter ip: ") # No need for str(), input() already returns a string
    ip_queue.put(ip)

# Calls function _ping with threading.
if __name__ == '__main__':
    numOfThreads = 2
    threadList = []

    # Place sentinels for each thread
    for i in range(numOfThreads):
        ip_queue.put(None)

    # Start each thread
    for i in range (numOfThreads):
        t = threading.Thread(target=_ping, args=(ip_queue,))
        t.start()
        threadList.append(t)

    # Usually good practice to wait for each thread
    for thread in threadList:
        thread.join()

回答2:

Suggest to use threadpool, first use pip install threadpool to install the module, then execute following code:

import threadpool

def ping(ip):
    print("Pinging ip: " + ip)


if __name__ == '__main__':
    ip_list = []

    for x in range (0, 4):
        ip = str(input("Please enter ip: "))
        ip_list.append(ip)

    numOfThreads = 2
    pool = threadpool.ThreadPool(numOfThreads)

    requests = threadpool.makeRequests(ping, ip_list)
    [pool.putRequest(req) for req in requests]
    pool.wait()

来源：https://stackoverflow.com/questions/51228018/how-to-have-a-thread-use-the-next-object-not-already-used

标签

python

multithreading

python-3.x