问题
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host
I am getting this error while reading a webpage in the following code from urllib.request import urlopen as uReq
from bs4 import BeautifulSoup as soup
myurl = 'https://www.amazon.in/s?k=graphics+card&ref=nb_sb_noss_2'
uClient =uReq(myurl)
回答1:
passing a useragent
header seems to solve the issue.
try something like this:
from urllib.request import urlopen as uReq, Request
from bs4 import BeautifulSoup as soup
headers = {"User-Agent": "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.3"}
my_url = 'https://www.amazon.in/s?k=graphics+card&ref=nb_sb_noss_2'
uClient = uReq(Request(url=my_url, headers=headers))
But do know that if the data you're trying to scrap is dynamic, bs4
wouldn't be of much help. consider using pyppeteer
or selenium
, etc.. for that.
来源:https://stackoverflow.com/questions/62462946/connectionreseterror-winerror-10054-an-existing-connection-was-forcibly-close