Python: requests.get, iterating url in a loop

Question

I am trying to get information from stats.nba.com by iterating requests.get(url) in a for loop, where the url changes at every iteration. If I just iterate it once it works

Answer 1

The website limits requests per second, so you'll need to include specific request headers or put a delay in your script (the first option being the quickest and likely most reliable of the two).

Headers Method:

'''
add under team_id = 1610612737
'''

HEADERS = {'user-agent': ('Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5)'
                          'AppleWebKit/537.36 (KHTML, like Gecko)'
                          'Chrome/45.0.2454.101 Safari/537.36'),
                          'referer': 'http://stats.nba.com/scores/'}

Then add this to your response get:

response = requests.get(url, headers=HEADERS)

*You shouldn't need to have a delay in your script at all if you use this method.

Delay Method:

import time
time.sleep(10) # delays for 10 seconds (put in your loop)

Seems like hit or miss using a delay, so I'd not recommend using unless absolutely necessary.