Asynchronously get and store images in python

Question

The following code is a sample of non-asynchronous code, is there any way to get the images asynchronously?

import urllib
for x in range(0,10):
        urllib.ur         


        

Answer 1

You don't need any third party library. Just create a thread for every request, start the threads, and then wait for all of them to finish in the background, or continue your application while the images are being downloaded.

import threading

results = []
def getter(url, dest):
   results.append(urllib.urlretreave(url, dest))

threads = []
for x in range(0,10):
    t = threading.Thread(target=getter, args=('http://test.com/file %s.png' % x,
                                              'temp/file %s.png' % x))
    t.start()
    threads.append(t)
# wait for all threads to finish
# You can continue doing whatever you want and
# join the threads when you finally need the results.
# They will fatch your urls in the background without
# blocking your main application.
map(lambda t: t.join(), threads)

Optionally you can create a thread pool that will get urls and dests from a queue.

If you're using Python 3 it's already implemented for you in the futures module.