ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host - getting this error

问题

ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host

I am getting this error while reading a webpage in the following code from urllib.request import urlopen as uReq

from bs4 import BeautifulSoup as soup

myurl = 'https://www.amazon.in/s?k=graphics+card&ref=nb_sb_noss_2'

uClient =uReq(myurl)

回答1:

passing a useragent header seems to solve the issue.

try something like this:

from urllib.request import urlopen as uReq, Request

from bs4 import BeautifulSoup as soup

headers = {"User-Agent": "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.3"}

my_url = 'https://www.amazon.in/s?k=graphics+card&ref=nb_sb_noss_2'

uClient = uReq(Request(url=my_url, headers=headers))

But do know that if the data you're trying to scrap is dynamic, bs4 wouldn't be of much help. consider using pyppeteer or selenium, etc.. for that.

来源：https://stackoverflow.com/questions/62462946/connectionreseterror-winerror-10054-an-existing-connection-was-forcibly-close