urllib.error.URLError: | 易学教程

问题

Hello I am trying to learn web scraping. I installed Anaconda3 in Windows 10. Conda version 4.5.12. Python version 3.7.1.

I wrote following script which produces the mentioned error.

import bs4
from bs4 import BeautifulSoup as soup
from urllib.request import urlopen as request
with request('https://google.com') as response:
    page_html = response.read()
page_soup = soup(page_html, "html.parser")
print(page_soup)

The error was from this line:

with request('https://google.com') as response:
...
...
raise URLError('unknown url type: %s' % type)
urllib.error.URLError: <urlopen error unknown url type: https>

However, when I opened up my Anaconda prompt and executed line by line from above script it worked flawlessly.
Can Anyone help me understand what went wrong? How can I make the script run from the console without getting this error?

回答1:

I solved the problem by reinstalling the anaconda library. On the installation process I chose to set up the PATH variable although it was not recommended. ( So i did not set the PATH first time ) So, basically I ignored the recommendation. And After that it worked. I do not know exactly what caused the problem first time. But now it is working.

回答2:

I had a similar problem installing the emsdk with Anaconda installed with the recommended settings.

I solved this way:

Open Anaconda Navigator > Click Environments > Select base(root) > Click the play icon > Select Open Terminal > Navigate to the script folder > Run the script

回答3:

May be this works,

response= request('https://google.com')
page_html = response.read() if response else ""
page_soup = soup(page_html, "html.parser")
print(page_soup)

回答4:

import ssl
ssl._create_default_https_context = ssl._create_unverified_context

This is the way to allow unverified SSL

回答5:

Try the below code:

import bs4
from bs4 import BeautifulSoup as soup
from urllib.request import urlopen, Request
response = urlopen('https://www.google.com')
page_html = response.read()
page_soup = soup(page_html, "html.parser")
print(page_soup)

来源：https://stackoverflow.com/questions/54926396/urllib-error-urlerror-urlopen-error-unknown-url-type-https

标签

python

web-scraping

Anaconda

urllib