Python urllib2: Cannot assign requested address

Question

I am sending thousands of requests using urllib2 with proxies. I have received many of the following error on execution:

urlopen error [Errno 9         


        

Answer 1

I have had a similar issue but was using POST command using python's request library though!! To make it worse, I used multiprocessing over each executor to post to a server. So thousands of connections created in seconds that took few seconds each to change the state from TIME_WAIT and release the ports for the next set of connections.

Out of all the available solutions available over the internet that speak of disabling keep-alive, using with request.Session() et al, I found this answer to be working which makes use of 'Connection' : 'close' configuration as header parameter. You may need to put the header content in a separte line outside the post command though.

headers = {
        'Connection': 'close'
}
with requests.Session() as session:
response = session.post('https://xx.xxx.xxx.x/xxxxxx/x', headers=headers, files=files, verify=False)
results = response.json()
print results

Just give it a try with request library.