Python requests/urllib3 raises 'httplib.BadStatusLine' error if called many times?

Question

I have a python program which utilizes python requests. I want to be able to run this program many times sequentially. The code can execute on its own, and run without any errors. However, when I try to run it 100 times, it will raise this error eventually:

ConnectionError: HTTPConnectionPool(host='192.168.100.1', port=80): Max retries exceeded with url: 'command' (Caused by <class 'httplib.BadStatusLine'>: '')

There are many different commands that are called, and is not always the same command. I have put in a delay in between the GET requests, so I don't think that because it is getting slammed with requests, it freezes up. (Although when I take out the delay, the error happens a lot more often).

Any ideas?! Thanks.

Answer 1

This is almost certainly either the server misbehaving or the connection being pre-emptively closed. Could you run it 99 times (assuming your 100 number is accurate) and on the 100th time edit the file to do:

import pdb

# before line with the call to requests
pdb.set_trace()

And then follow the stack trace through the HTTPAdapter and into urllib3 and look at the response urllib3 gets?