Asynchronously get and store images in python

Question

The following code is a sample of non-asynchronous code, is there any way to get the images asynchronously?

import urllib
for x in range(0,10):
        urllib.ur         


        

Answer 1

Something like this should help you

import grequests
urls = ['url1', 'url2', ....] # this should be the list of urls

    requests = (grequests.get(u) for u in urls)
    responses = grequests.map(requests)
    for response in responses:
        if 199 < response.status_code < 400:
             name = generate_file_name()    # generate some name for your image file with extension like example.jpg
             with open(name, 'wb') as f:    # or save to S3 or something like that
                  f.write(response.content)

Here only the downloading of images would be parallel but writing each image content to a file would be sequential so you can create a thread or do something else to make it parallel or asynchronous