如何从python中的请求获取响应SSL证书?

时间:2013-06-03 18:37:19

标签: python http https request python-requests

尝试从requests中的响应中获取SSL证书。

这样做的好方法是什么?

6 个答案:

答案 0 :(得分:13)

requests故意包装像这样的低级别的东西。通常,您唯一想做的就是verify that the certs are valid。要做到这一点,只需通过verify=True。如果你想使用非标准的cacert包,你也可以传递它。例如:

resp = requests.get('https://example.com', verify=True, cert=['/path/to/my/ca.crt'])

此外,requests主要是围绕其他库的一组包装器,主要是urllib3和stdlib的http.client(或者,对于2.x,httplib)和{ {3}}。

有时,答案只是为了获得较低级别的对象(例如,resp.rawurllib3.response.HTTPResponse),但在许多情况下这是不可能的。

这就是其中一个案例。唯一看到证书的对象是http.client.HTTPSConnection(或urllib3.connectionpool.VerifiedHTTPSConnection,但这只是前者的子类)和ssl.SSLSocket,当时这些对象都不存在请求返回。 (正如名称connectionpool所暗示的那样,HTTPSConnection对象存储在池中,并且可以在完成后立即重用; SSLSocketHTTPSConnection的成员。)

因此,您需要修补内容,以便将数据复制到链中。它可能很简单:

HTTPResponse = requests.packages.urllib3.response.HTTPResponse
orig_HTTPResponse__init__ = HTTPResponse.__init__
def new_HTTPResponse__init__(self, *args, **kwargs):
    orig_HTTPResponse__init__(self, *args, **kwargs)
    try:
        self.peercert = self._connection.sock.getpeercert()
    except AttributeError:
        pass
HTTPResponse.__init__ = new_HTTPResponse__init__

HTTPAdapter = requests.adapters.HTTPAdapter
orig_HTTPAdapter_build_response = HTTPAdapter.build_response
def new_HTTPAdapter_build_response(self, request, resp):
    response = orig_HTTPAdapter_build_response(self, request, resp)
    try:
        response.peercert = resp.peercert
    except AttributeError:
        pass
    return response
HTTPAdapter.build_response = new_HTTPAdapter_build_response

那是未经测试的,所以没有保证;你可能需要修补更多。

此外,子类化和重写可能比monkeypatching更清晰(特别是因为HTTPAdapter被设计为子类)。

或者,更好的是,分叉urllib3requests,修改你的分叉,以及(如果你认为这是合法有用的)向上游提交拉请求。

无论如何,现在,从您的代码中,您可以执行此操作:

resp.peercert

这将为您提供包含'subject''subjectAltName'键的词典,由pyopenssl.WrappedSocket.getpeercert返回。如果您想要了解有关证书的更多信息,请尝试使用ssl来获取OpenSSL.crypto.X509对象。如果您想获得整个对等证书链,请参阅Christophe Vandeplas's variant of this answer

当然,您可能还需要传递验证证书所需的所有信息,但这更容易,因为它已经通过顶层。

答案 1 :(得分:7)

首先,abarnert's answer非常完整。在追踪拟议的Kalkranconnection-close问题时,我实际上发现peercert没有包含有关SSL证书的详细信息。

我在连接和套接字信息中进行了更深入的研究,并提取了self.sock.connection.get_peer_certificate()函数,该函数包含如下功能:

  • get_subject() for CN
  • get_notAfter()get_notBefore()的截止日期
  • get_serial_number()get_signature_algorithm()了解与加密有关的技术细节
  • ...

请注意,只有在系统上安装了pyopenssl时,这些选项才可用。在后台,urllib3使用pyopenssl(如果可用),否则使用标准库的ssl模块。以下显示的self.sock.connection属性仅在self.sockurllib3.contrib.pyopenssl.WrappedSocket时才存在,而在ssl.SSLSocket中则不存在。您可以将pyopensslpip install pyopenssl一起安装。

完成后,代码将变为:

import requests

HTTPResponse = requests.packages.urllib3.response.HTTPResponse
orig_HTTPResponse__init__ = HTTPResponse.__init__
def new_HTTPResponse__init__(self, *args, **kwargs):
    orig_HTTPResponse__init__(self, *args, **kwargs)
    try:
        self.peer_certificate = self._connection.peer_certificate
    except AttributeError:
        pass
HTTPResponse.__init__ = new_HTTPResponse__init__

HTTPAdapter = requests.adapters.HTTPAdapter
orig_HTTPAdapter_build_response = HTTPAdapter.build_response
def new_HTTPAdapter_build_response(self, request, resp):
    response = orig_HTTPAdapter_build_response(self, request, resp)
    try:
        response.peer_certificate = resp.peer_certificate
    except AttributeError:
        pass
    return response
HTTPAdapter.build_response = new_HTTPAdapter_build_response

HTTPSConnection = requests.packages.urllib3.connection.HTTPSConnection
orig_HTTPSConnection_connect = HTTPSConnection.connect
def new_HTTPSConnection_connect(self):
    orig_HTTPSConnection_connect(self)
    try:
        self.peer_certificate = self.sock.connection.get_peer_certificate()
    except AttributeError:
        pass
HTTPSConnection.connect = new_HTTPSConnection_connect

您将能够轻松访问结果:

r = requests.get('https://yourdomain.tld', timeout=0.1)
print('Expires on: {}'.format(r.peer_certificate.get_notAfter()))
print(dir(r.peer_certificate))

如果像我一样要忽略SSL证书警告,只需在文件顶部添加以下内容,而不进行SSL验证:

from requests.packages.urllib3.exceptions import InsecureRequestWarning
requests.packages.urllib3.disable_warnings(InsecureRequestWarning)

r = requests.get('https://yourdomain.tld', timeout=0.1, verify=False)
print(dir(r.peer_certificate))

答案 2 :(得分:3)

这虽然不是很好,但有效:

import requests

req = requests.get('https://httpbin.org')
pool = req.connection.poolmanager.connection_from_url('https://httpbin.org')
conn = pool.pool.get()
# get() removes it from the pool, so put it back in
pool.pool.put(conn)
print(conn.sock.getpeercert())

答案 3 :(得分:3)

感谢大家的精彩回答。

它帮助我解决了这个问题的答案:

How to add a custom CA Root certificate to the CA Store used by Python in Windows?

更新2019-02-12

  

请查看Cert Human: SSL Certificates for Humans,以https://github.com/neozenith/get-ca-py重写我的lifehackjim项目。

     

我已经存档了原始存储库。

独立代码段

#! /usr/bin/env python
# -*- coding: utf-8 -*-
"""
Get Certificates from a request and dump them.
"""

import argparse
import sys

import requests
from requests.packages.urllib3.exceptions import InsecureRequestWarning

requests.packages.urllib3.disable_warnings(InsecureRequestWarning)

"""
Inspired by the answers from this Stackoverflow question:
https://stackoverflow.com/questions/16903528/how-to-get-response-ssl-certificate-from-requests-in-python

What follows is a series of patching the low level libraries in requests.
"""

"""
https://stackoverflow.com/a/47931103/622276
"""

sock_requests = requests.packages.urllib3.contrib.pyopenssl.WrappedSocket


def new_getpeercertchain(self, *args, **kwargs):
    x509 = self.connection.get_peer_cert_chain()
    return x509


sock_requests.getpeercertchain = new_getpeercertchain

"""
https://stackoverflow.com/a/16904808/622276
"""

HTTPResponse = requests.packages.urllib3.response.HTTPResponse
orig_HTTPResponse__init__ = HTTPResponse.__init__


def new_HTTPResponse__init__(self, *args, **kwargs):
    orig_HTTPResponse__init__(self, *args, **kwargs)
    try:
        self.peercertchain = self._connection.sock.getpeercertchain()
    except AttributeError:
        pass


HTTPResponse.__init__ = new_HTTPResponse__init__

HTTPAdapter = requests.adapters.HTTPAdapter
orig_HTTPAdapter_build_response = HTTPAdapter.build_response


def new_HTTPAdapter_build_response(self, request, resp):
    response = orig_HTTPAdapter_build_response(self, request, resp)
    try:
        response.peercertchain = resp.peercertchain
    except AttributeError:
        pass
    return response


HTTPAdapter.build_response = new_HTTPAdapter_build_response

"""
Attempt to wrap in a somewhat usable CLI
"""


def cli(args):
    parser = argparse.ArgumentParser(description="Request any URL and dump the certificate chain")
    parser.add_argument("url", metavar="URL", type=str, nargs=1, help="Valid https URL to be handled by requests")

    verify_parser = parser.add_mutually_exclusive_group(required=False)
    verify_parser.add_argument("--verify", dest="verify", action="store_true", help="Explicitly set SSL verification")
    verify_parser.add_argument(
        "--no-verify", dest="verify", action="store_false", help="Explicitly disable SSL verification"
    )
    parser.set_defaults(verify=True)

    return vars(parser.parse_args(args))


def dump_pem(cert, outfile="ca-chain.crt"):
    """Use the CN to dump certificate to PEM format"""
    PyOpenSSL = requests.packages.urllib3.contrib.pyopenssl
    pem_data = PyOpenSSL.OpenSSL.crypto.dump_certificate(PyOpenSSL.OpenSSL.crypto.FILETYPE_PEM, cert)
    issuer = cert.get_issuer().get_components()

    print(pem_data.decode("utf-8"))

    with open(outfile, "a") as output:
        for part in issuer:
            output.write(part[0].decode("utf-8"))
            output.write("=")
            output.write(part[1].decode("utf-8"))
            output.write(",\t")
        output.write("\n")
        output.write(pem_data.decode("utf-8"))


if __name__ == "__main__":
    cli_args = cli(sys.argv[1:])

    url = cli_args["url"][0]
    req = requests.get(url, verify=cli_args["verify"])
    for cert in req.peercertchain:
        dump_pem(cert)

答案 4 :(得分:2)

首先,abarnert's answer非常完整

但我想补充一点,在你正在寻找对等证书链的情况下,你需要修补另一段代码

import requests
sock_requests = requests.packages.urllib3.contrib.pyopenssl.WrappedSocket
def new_getpeercertchain(self,*args, **kwargs):
    x509 = self.connection.get_peer_cert_chain()
    return x509
sock_requests.getpeercertchain = new_getpeercertchain

之后你可以用一种非常类似的方式把它称为接受的答案

HTTPResponse = requests.packages.urllib3.response.HTTPResponse
orig_HTTPResponse__init__ = HTTPResponse.__init__
def new_HTTPResponse__init__(self, *args, **kwargs):
    orig_HTTPResponse__init__(self, *args, **kwargs)
    try:
        self.peercertchain = self._connection.sock.getpeercertchain()
    except AttributeError:
        pass
HTTPResponse.__init__ = new_HTTPResponse__init__

HTTPAdapter = requests.adapters.HTTPAdapter
orig_HTTPAdapter_build_response = HTTPAdapter.build_response
def new_HTTPAdapter_build_response(self, request, resp):
    response = orig_HTTPAdapter_build_response(self, request, resp)
    try:
        response.peercertchain = resp.peercertchain
    except AttributeError:
        pass
    return response
HTTPAdapter.build_response = new_HTTPAdapter_build_response

您将获得resp.peercertchain,其中包含tupleOpenSSL.crypto.X509个对象

答案 5 :(得分:0)

要检索证书的详细信息(例如CN和有效日期),以下根据此example改编的脚本效果很好。它也避免了一些我认为是由于请求和urllib3版本不正确/不兼容而导致的错误:“ AttributeError:'SSLSocket'对象没有属性'connection'”和“ AttributeError:'VerifiedHTTPSConnection'对象没有属性'peer_certificate' “

from OpenSSL.SSL import Connection, Context, SSLv3_METHOD, TLSv1_2_METHOD
from datetime import datetime, time
import socket
host = 'www.google.com'
try:
    try:
        ssl_connection_setting = Context(SSLv3_METHOD)
    except ValueError:
        ssl_connection_setting = Context(TLSv1_2_METHOD)
    ssl_connection_setting.set_timeout(5)
    with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
        s.connect((host, 443))
        c = Connection(ssl_connection_setting, s)
        c.set_tlsext_host_name(str.encode(host))
        c.set_connect_state()
        c.do_handshake()
        cert = c.get_peer_certificate()
        print("Is Expired: ", cert.has_expired())
        print("Issuer: ", cert.get_issuer())
        subject_list = cert.get_subject().get_components()
        cert_byte_arr_decoded = {}
        for item in subject_list:
            cert_byte_arr_decoded.update({item[0].decode('utf-8'): item[1].decode('utf-8')})
        print(cert_byte_arr_decoded)
        if len(cert_byte_arr_decoded) > 0:
            print("Subject: ", cert_byte_arr_decoded)
        if cert_byte_arr_decoded["CN"]:
            print("Common Name: ", cert_byte_arr_decoded["CN"])
        end_date = datetime.strptime(str(cert.get_notAfter().decode('utf-8')), "%Y%m%d%H%M%SZ")
        print("Not After (UTC Time): ", end_date)
        diff = end_date - datetime.now()
        print('Summary: "{}" SSL certificate expires on {} i.e. {} days.'.format(host, end_date, diff.days))
        c.shutdown()
        s.close()
except:
    print("Connection to {} failed.".format(host))  

此脚本需要Python 3和pyOpenSSL。