I have a script that crawls a website. Untill today it ran perfect, However it does not do so now.
it give sme the following error:
Connection Abort
Try increasing the timeout
parameter of your requests.get
method :
requests.get(functionurl, headers=headers, timeout=5)
But the odds are that your script is being blocked by the server to prevent scrapping attempts . If this is the case you can try faking a web browser by setting appropriate headers .
{"User-Agent": "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.8) Gecko/20100722 Firefox/3.6.8 GTB7.1 (.NET CLR 3.5.30729)", "Referer": "http://example.com"}