Understanding requests versus grequests

Question

I\'m working with a process which is basically as follows:

1. Take some list of urls.
2. Get a Response object from each.
3. Create a BeautifulSoup object

Answer 1

I do not know the exact reason for the observed behavior with .map(). However, using the .imap() function with size=1 always returned a 'Response 200' for my few minutes testing. Here is the code snipet:

rs = (grequests.get(u) for u in [BASE.format(t) for t in tickers])
rsm_iterator = grequests.imap(rs, exception_handler=exception_handler, size=1)
rsm_list = [r for r in rsm_iterator]
print(rsm_list)

And if you don't want to wait for all requests to finish before working on their answers, you can do this like so:

rs = (grequests.get(u) for u in [BASE.format(t) for t in tickers])
rsm_iterator = grequests.imap(rs, exception_handler=exception_handler, size=1)
for r in rsm_iterator:
    print(r)