I am trying to get data from the web using python. I imported urllib.request package for it but while executing, I get error:
certificate verify failed: unab
I recently had this issue while connecting to MongoDB Atlas. I updated to the latest certifi
python package and it works now.
(python 3.8, upgraded to certifi 2020.4.5.1, previously certifi version 2019.11.28)
This page is the top google hit for "certificate verify failed: unable to get local issuer certificate", so while this doesn't directly answer the original question, below is a fix for a problem with the same symptom. I ran into this while trying to add TLS to an xmlrpc service. This requires use of the fairly low-level ssl.SSLContext
class. The error indicates that a certificate is missing. The fix was to do several things when constructing SSLContext
objects:
First, in the client:
def get_client():
context = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
# Load the default certs:
context.load_default_certs()
# Optionally, install the intermediate certs.
# This _should_ be handled by the server, but
# maybe helpful in some cases.
# context.load_verify_locations('path/to/ca_bundle.crt')
return xmlrpc.client.ServerProxy('https://server.yourdomain.com/', context=context)
In the server, you need to install the intermediate certs in the context:
class SecureXMLRPCServer(socketserver.TCPServer,
xmlrpc.server.SimpleXMLRPCDispatcher):
# https://gist.github.com/monstermunchkin/1100226
allow_reuse_address = True
def __init__(self, addr, certfile, keyfile=None,
ca_bundle=None,
requestHandler=xmlrpc.server.SimpleXMLRPCRequestHandler,
logRequests=True, allow_none=False, encoding=None,
bind_and_activate=True, ssl_version=ssl.PROTOCOL_TLSv1_2):
self.logRequests = logRequests
# create an SSL context
self.context = ssl.SSLContext(ssl_version)
self.context.load_default_certs()
# The server is the correct place to load the intermediate CA certificates:
self.context.load_verify_locations(ca_bundle)
self.context.load_cert_chain(certfile=certfile, keyfile=keyfile)
xmlrpc.server.SimpleXMLRPCDispatcher.__init__(self, allow_none,
encoding)
# call TCPServer constructor
socketserver.TCPServer.__init__(self, addr, requestHandler,
bind_and_activate)
if fcntl is not None and hasattr(fcntl, 'FD_CLOEXEC'):
flags = fcntl.fcntl(self.fileno(), fcntl.F_GETFD)
flags |= fcntl.FD_CLOEXEC
fcntl.fcntl(self.fileno(), fcntl.F_SETFD, flags)
def get_request(self):
newsocket, fromaddr = self.socket.accept()
# create an server-side SSL socket
sslsocket = self.context.wrap_socket(newsocket, server_side=True)
return sslsocket, fromaddr
For me the problem was that I was setting REQUESTS_CA_BUNDLE
in my .bash_profile
/Users/westonagreene/.bash_profile:
...
export REQUESTS_CA_BUNDLE=/usr/local/etc/openssl/cert.pem
...
Once I set REQUESTS_CA_BUNDLE
to blank (i.e. removed from .bash_profile), requests
worked again.
export REQUESTS_CA_BUNDLE=""
The problem only exhibited when executing python requests
via a CLI (Command Line Interface). If I ran requests.get(URL, CERT)
it resolved just fine.
Mac OS Catalina (10.15.6).
Pyenv of 3.6.11.
Error message I was getting: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1056)
This answer elsewhere: https://stackoverflow.com/a/64152045/4420657
I would like to provide a reference. I use cmd + space, then type Install Certificates.command
, and then press Enter. After a short while, the command line interface pops up to start the installation.
-- removing any existing file or link
-- creating symlink to certifi certificate bundle
-- setting permissions
-- update complete
Finally, it fixes the errors.
I hit the same issue on OSX, while my code was totally fine on Linux, and you gave the answer in your question!
After inspecting the file you pointed to /Applications/Python 3.7/Install Certificates.command
, it turned out that what this command replaces the root certificates of the default Python installation with the ones shipped through the certifi
package.
certifi
is a set of root certificates. Each SSL certificate relies a chain of trust: you trust one specific certificate because you trust the parent of that certificate, for which you trust the parent, etc. At some point, there is no "parent" and those are "root" certificates. For those, there is no other solution than bundling commonly trusted root certificates (usually big trust companies like eg. "DigiCert").
You can for instance see the root certificates in your browser security settings (for instance for Firefox->Preference->Privacy and security->view certificates->Authorities).
Coming back to the initial problem, and prior to running the .command
file, executing this returns for me an empty list on a clean installation:
import os
import ssl
openssl_dir, openssl_cafile = os.path.split(
ssl.get_default_verify_paths().openssl_cafile)
# no content in this folder
os.listdir(openssl_dir)
# non existent file
print(os.path.exists(openssl_cafile))
This means that there is no default certificate authority for the Python installation on OSX. A possible default is exactly the one provided by the certifi
package.
After that, you just can create an SSL context that has the proper default as the following (certifi.where()
gives the location of a certificate authority):
import platform
# ...
ssl_context = ssl.SSLContext(ssl.PROTOCOL_TLS)
ssl_context.verify_mode = ssl.CERT_REQUIRED
ssl_context.check_hostname = True
ssl_context.load_default_certs()
if platform.system().lower() == 'darwin':
import certifi
ssl_context.load_verify_locations(
cafile=os.path.relpath(certifi.where()),
capath=None,
cadata=None)
and make request to an url
from python like this:
import urllib
# previous context
https_handler = urllib.request.HTTPSHandler(context=ssl_context)
opener = urllib.request.build_opener(https_handler)
ret = opener.open(url, timeout=2)
This worked in all OS:
import ssl
import certifi
urlopen(request, context=ssl.create_default_context(cafile=certifi.where()))