How to use ssl client certificate (p12) with Scrapy?

我与影子孤独终老i 提交于 2021-02-18 10:42:07

问题


I need to use client certificate file in format p12(PKCS12) to talk to a webserver with scrapy, is there a way to do that ?


回答1:


I can't offer you a tested and complete solution here, but I know a few places where some adjustments might give you what you need.

The starting point is scrapy's ContextFactory object which defines the SSL/TLS configuration. The standard implementation ScrapyClientContextFactory doesn't use client certificates and also doesn't do any server certificate verification, it just accepts any certificate. (More details)

When looking into the source code however you see the alternative BrowserLikeContextFactory is creating an optionsForClientTLS object.

This object can also take a clientCertificate parameter to authenticate to the server. (Details)

So in theory you need to subclass BrowserLikeContextFactory, write there your own creatorForNetloc method and make it create optionsForClientTLS that also have a clientCertificate

In a gist:

@implementer(IPolicyForHTTPS)
class ClientCertContextFactory(BrowserLikeContextFactory):

    def creatorForNetloc(self, hostname, port):
        with open('yourcert.pem') as keyAndCert:
            myClientCert = twisted.internet.ssl.PrivateCertificate.load(keyAndCert.read())
        return optionsForClientTLS(hostname.decode("ascii"),
                                   trustRoot=platformTrust(),
                                   clientCertificate=myClientCert,
                                   extraCertificateOptions={
                                        'method': self._ssl_method,
                                   })

Activating the context factory in settings.py:

DOWNLOADER_CLIENTCONTEXTFACTORY = 'your.package.ClientCertContextFactory'

According to the docs twisted.internet.ssl.PrivateCertificate can only load pem or asn.1 format keys, means you will have to convert your key into pem format:

openssl pkcs12 -in client_ssl.pfx -out client_ssl.pem -clcerts

(Borrowed from Converting pfx to pem using openssl)

Update Conversion for PKCS12 files in p12 format:

openssl pkcs12 -in client_cert.p12 -out client_cert.pem -clcerts


来源:https://stackoverflow.com/questions/43739207/how-to-use-ssl-client-certificate-p12-with-scrapy

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!