为什么urllib3 / idna抱怨X509证书中的通配符?我如何解决它?

时间:2016-09-15 22:16:49

标签: python-2.7 wildcard x509certificate urllib3 treasure-data

我在虚拟环境下运行以下内容:

setuptools==26.1.1
td-client==0.5.0
urllib3[secure]==1.17

如果我不使用它,则使用urllib3由于SNIMissingWarning:

http://urllib3.readthedocs.io/en/latest/user-guide.html#ssl-py2

我还在用户指南中列出了所需的系统包:

dpkg -l | grep build-essential
ii  build-essential                      11.6ubuntu6                         amd64        Informational list of build-essential packages
dpkg -l | grep libssl-dev
ii  libssl-dev:amd64                     1.0.1f-1ubuntu2.7                   amd64        Secure Sockets Layer toolkit - development files
dpkg -l | grep libffi-dev
ii  libffi-dev:amd64                     3.1~rc1+r3.0.13-12                  amd64        Foreign Function Interface library (development files)
dpkg -l | grep "\spython-dev\s"
ii  python-dev                           2.7.5-5ubuntu3                      amd64        header files and a static library for Python (default)

这是我的代码:

#!./venv/bin/python

from argparse import ArgumentParser
import logging
import tdclient
import urllib3.contrib.pyopenssl
import certifi
import urllib3

def main():
    for key in logging.Logger.manager.loggerDict:
            if key == "tdclient.api": continue
            if key == "tdclient": continue
            logging.getLogger(key).setLevel(logging.DEBUG)
    urllib3.contrib.pyopenssl.inject_into_urllib3()
    http = urllib3.PoolManager(cert_reqs='CERT_REQUIRED', ca_certs=certifi.where())
    key = getKey()
    listTdTables(key)

def listTdTables(key):
    with tdclient.Client(key) as client:
            for db in client.databases():
                    for table in db.tables():
                            print(table.db_name)
                            print(table.table_name)
                            print(table.count)

def getKey():
    parser = ArgumentParser()
    parser.add_argument("-k", "--key", dest="key", help="key", required=True)
    options = parser.parse_args()
    return options.key

if __name__ == "__main__":
    main()

Stacktrace我得到了:

Traceback (most recent call last):
  File "./akamai_pusher.py", line 35, in <module>
    main()
  File "./akamai_pusher.py", line 18, in main
    listTdTables(key)
  File "./akamai_pusher.py", line 22, in listTdTables
    for db in client.databases():
  File "/mnt/home/marc/repos/akamai_pusher/versions/1473977463594675887/venv/local/lib/python2.7/site-packages/tdclient/client.py", line 81, in databases
    databases = self.api.list_databases()
  File "/mnt/home/marc/repos/akamai_pusher/versions/1473977463594675887/venv/local/lib/python2.7/site-packages/tdclient/database_api.py", line 21, in list_databases
    with self.get("/v3/database/list") as res:
  File "/mnt/home/marc/repos/akamai_pusher/versions/1473977463594675887/venv/local/lib/python2.7/site-packages/tdclient/api.py", line 174, in get
    response = self.send_request("GET", url, fields=params, headers=headers, decode_content=True, preload_content=False)
  File "/mnt/home/marc/repos/akamai_pusher/versions/1473977463594675887/venv/local/lib/python2.7/site-packages/tdclient/api.py", line 345, in send_request
    return self.http.request(method, url, fields=fields, **kwargs)
  File "/mnt/home/marc/repos/akamai_pusher/versions/1473977463594675887/venv/local/lib/python2.7/site-packages/urllib3/request.py", line 66, in request
    **urlopen_kw)
  File "/mnt/home/marc/repos/akamai_pusher/versions/1473977463594675887/venv/local/lib/python2.7/site-packages/urllib3/request.py", line 87, in request_encode_url
    return self.urlopen(method, url, **extra_kw)
  File "/mnt/home/marc/repos/akamai_pusher/versions/1473977463594675887/venv/local/lib/python2.7/site-packages/urllib3/poolmanager.py", line 244, in urlopen
    response = conn.urlopen(method, u.request_uri, **kw)
  File "/mnt/home/marc/repos/akamai_pusher/versions/1473977463594675887/venv/local/lib/python2.7/site-packages/urllib3/connectionpool.py", line 594, in urlopen
    chunked=chunked)
  File "/mnt/home/marc/repos/akamai_pusher/versions/1473977463594675887/venv/local/lib/python2.7/site-packages/urllib3/connectionpool.py", line 350, in _make_request
    self._validate_conn(conn)
  File "/mnt/home/marc/repos/akamai_pusher/versions/1473977463594675887/venv/local/lib/python2.7/site-packages/urllib3/connectionpool.py", line 833, in _validate_conn
    conn.connect()
  File "/mnt/home/marc/repos/akamai_pusher/versions/1473977463594675887/venv/local/lib/python2.7/site-packages/urllib3/connection.py", line 324, in connect
    cert = self.sock.getpeercert()
  File "/mnt/home/marc/repos/akamai_pusher/versions/1473977463594675887/venv/local/lib/python2.7/site-packages/urllib3/contrib/pyopenssl.py", line 312, in getpeercert
    'subjectAltName': get_subj_alt_name(x509)
  File "/mnt/home/marc/repos/akamai_pusher/versions/1473977463594675887/venv/local/lib/python2.7/site-packages/urllib3/contrib/pyopenssl.py", line 185, in get_subj_alt_name
    for name in ext.get_values_for_type(x509.DNSName)
  File "/mnt/home/marc/repos/akamai_pusher/versions/1473977463594675887/venv/local/lib/python2.7/site-packages/urllib3/contrib/pyopenssl.py", line 141, in _dnsname_to_stdlib
    name = idna.encode(name)
  File "/mnt/home/marc/repos/akamai_pusher/versions/1473977463594675887/venv/local/lib/python2.7/site-packages/idna/core.py", line 355, in encode
    result.append(alabel(label))
  File "/mnt/home/marc/repos/akamai_pusher/versions/1473977463594675887/venv/local/lib/python2.7/site-packages/idna/core.py", line 276, in alabel
    check_label(label)
  File "/mnt/home/marc/repos/akamai_pusher/versions/1473977463594675887/venv/local/lib/python2.7/site-packages/idna/core.py", line 253, in check_label
    raise InvalidCodepoint('Codepoint {0} at position {1} of {2} not allowed'.format(_unot(cp_value), pos+1, repr(label)))
idna.core.InvalidCodepoint: Codepoint U+002A at position 1 of u'*' not allowed

x509证书urllib3 / idna的详细信息正在开启(通过略微修改的getpeercert):

**************************************************************************************************
x509
**************************************************************************************************
Extensions:
    0: CA:FALSE
    1: TLS Web Server Authentication, TLS Web Client Authentication
    2: Digital Signature, Key Encipherment
    3: 
Full Name:
  URI:http://crl.godaddy.com/gdig2s1-39.crl

    4: Policy: 2.16.840.1.114413.1.7.23.1
  CPS: http://certificates.godaddy.com/repository/

    5: OCSP - URI:http://ocsp.godaddy.com/
CA Issuers - URI:http://certificates.godaddy.com/repository/gdig2.crt

    6: keyid:40:C2:BD:27:8E:CC:34:83:30:A2:33:D7:FB:6C:B3:F0:B4:2C:80:CE

    7: DNS:*.treasuredata.com, DNS:treasuredata.com
    8: DE:83:BE:82:93:41:DD:66:0C:57:2C:31:48:CD:A9:D8:4E:9F:7A:D3
Issuer: <X509Name object '/C=US/ST=Arizona/L=Scottsdale/O=GoDaddy.com, Inc./OU=http://certs.godaddy.com/repository//CN=Go Daddy Secure Certificate Authority - G2'>
Not After: 20170120100704Z
Not Before: 20140410060603Z
Pubkey:
    bits: 2048
    type: 6
    key: <cryptography.hazmat.backends.openssl.rsa._RSAPublicKey object at 0x7fede525bdd0>
sha1_fingerprint: 8C:55:93:A7:7A:83:D5:51:7F:8F:FB:43:A3:AC:04:31:F5:66:E0:72
sha256_fingerprint: 97:85:1A:38:D0:C4:48:3E:D7:B9:96:C4:BC:94:C6:EC:6F:5A:47:86:7E:78:48:CC:06:B6:AB:29:C4:3F:BE:67
Serial Number: 12259260892195744
Signature Algorith: sha256WithRSAEncryption
Subject: <X509Name object '/OU=Domain Control Validated/CN=*.treasuredata.com'>
Version: 2
Has Expired : False
Subject Name Hash: 28698744
**************************************************************************************************

https://certificatedetails.com/40c2bd278ecc348330a233d7fb6cb3f0b42c80ce/2b8dbb9ab02ba0/.treasuredata.com

新例子,sans tdclient:

首先,我简化了我的依赖项:

setuptools==26.1.1
urllib3[secure]==1.17

然后是代码:

#!./venv/bin/python

import urllib3.contrib.pyopenssl
import certifi
import urllib3
import sys
import traceback

def main():
    urllib3.contrib.pyopenssl.inject_into_urllib3()
    http = urllib3.PoolManager(cert_reqs='CERT_REQUIRED', ca_certs=certifi.where())
    try:
            print("*********** https://www.google.com ***********")
            print(http.request('GET', 'https://www.google.com', timeout=4.0))
            print("********** https://www.facebook.com **********")
            print(http.request('GET', 'https://www.facebook.com', timeout=4.0))
            print("**********************************************")
    except:
            traceback.print_exc(limit=None, file=sys.stderr)
    finally:
            print("**********************************************")

if __name__ == "__main__":
    main()

我得到了以下输出:

*********** https://www.google.com ***********
Traceback (most recent call last):
  File "./akamai_pusher.py", line 14, in main
    print(http.request('GET', 'https://www.google.com', timeout=4.0))
  File "/mnt/home/marc/repos/akamai_pusher/versions/1474000948101039366/venv/local/lib/python2.7/site-packages/urllib3/request.py", line 66, in request
    **urlopen_kw)
  File "/mnt/home/marc/repos/akamai_pusher/versions/1474000948101039366/venv/local/lib/python2.7/site-packages/urllib3/request.py", line 87, in request_encode_url
    return self.urlopen(method, url, **extra_kw)
  File "/mnt/home/marc/repos/akamai_pusher/versions/1474000948101039366/venv/local/lib/python2.7/site-packages/urllib3/poolmanager.py", line 244, in urlopen
    response = conn.urlopen(method, u.request_uri, **kw)
  File "/mnt/home/marc/repos/akamai_pusher/versions/1474000948101039366/venv/local/lib/python2.7/site-packages/urllib3/connectionpool.py", line 624, in urlopen
    raise SSLError(e)
SSLError: bad handshake: Error([('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')],)
**********************************************

我得到了here的建议,我应该将certifi的版本降级到2015.4.28:

setuptools==26.1.1
certifi==2015.4.28
urllib3[secure]==1.17

适用于Google链接。但不是因为facebook链接,最终出现了与td-client相同的问题:

*********** https://www.google.com ***********
<urllib3.response.HTTPResponse object at 0x7fc1d8942590>
********** https://www.facebook.com **********
Traceback (most recent call last):
  File "./akamai_pusher.py", line 16, in main
    print(http.request('GET', 'https://www.facebook.com', timeout=4.0))
  File "/mnt/home/marc/repos/akamai_pusher/versions/1474001584864131764/venv/local/lib/python2.7/site-packages/urllib3/request.py", line 66, in request
    **urlopen_kw)
  File "/mnt/home/marc/repos/akamai_pusher/versions/1474001584864131764/venv/local/lib/python2.7/site-packages/urllib3/request.py", line 87, in request_encode_url
    return self.urlopen(method, url, **extra_kw)
  File "/mnt/home/marc/repos/akamai_pusher/versions/1474001584864131764/venv/local/lib/python2.7/site-packages/urllib3/poolmanager.py", line 244, in urlopen
    response = conn.urlopen(method, u.request_uri, **kw)
  File "/mnt/home/marc/repos/akamai_pusher/versions/1474001584864131764/venv/local/lib/python2.7/site-packages/urllib3/connectionpool.py", line 594, in urlopen
    chunked=chunked)
  File "/mnt/home/marc/repos/akamai_pusher/versions/1474001584864131764/venv/local/lib/python2.7/site-packages/urllib3/connectionpool.py", line 350, in _make_request
    self._validate_conn(conn)
  File "/mnt/home/marc/repos/akamai_pusher/versions/1474001584864131764/venv/local/lib/python2.7/site-packages/urllib3/connectionpool.py", line 833, in _validate_conn
    conn.connect()
  File "/mnt/home/marc/repos/akamai_pusher/versions/1474001584864131764/venv/local/lib/python2.7/site-packages/urllib3/connection.py", line 324, in connect
    cert = self.sock.getpeercert()
  File "/mnt/home/marc/repos/akamai_pusher/versions/1474001584864131764/venv/local/lib/python2.7/site-packages/urllib3/contrib/pyopenssl.py", line 312, in getpeercert
    'subjectAltName': get_subj_alt_name(x509)
  File "/mnt/home/marc/repos/akamai_pusher/versions/1474001584864131764/venv/local/lib/python2.7/site-packages/urllib3/contrib/pyopenssl.py", line 185, in get_subj_alt_name
    for name in ext.get_values_for_type(x509.DNSName)
  File "/mnt/home/marc/repos/akamai_pusher/versions/1474001584864131764/venv/local/lib/python2.7/site-packages/urllib3/contrib/pyopenssl.py", line 141, in _dnsname_to_stdlib
    name = idna.encode(name)
  File "/mnt/home/marc/repos/akamai_pusher/versions/1474001584864131764/venv/local/lib/python2.7/site-packages/idna/core.py", line 355, in encode
    result.append(alabel(label))
  File "/mnt/home/marc/repos/akamai_pusher/versions/1474001584864131764/venv/local/lib/python2.7/site-packages/idna/core.py", line 276, in alabel
    check_label(label)
  File "/mnt/home/marc/repos/akamai_pusher/versions/1474001584864131764/venv/local/lib/python2.7/site-packages/idna/core.py", line 253, in check_label
    raise InvalidCodepoint('Codepoint {0} at position {1} of {2} not allowed'.format(_unot(cp_value), pos+1, repr(label)))
InvalidCodepoint: Codepoint U+002A at position 1 of u'*' not allowed
**********************************************

1 个答案:

答案 0 :(得分:0)

所以我发现了这个问题:urllib3 v1.17(至少在python 2.7下)。我降级到v1.16和中提琴,代码有效。 (v1.17刚刚在9天前发布)以下是对我有用的依赖项:

setuptools==26.1.1
certifi==2015.4.28
urllib3[secure]==1.16
td-client==0.5.0

我认为this可能与它有关:

  

从用于PyOpenSSL的依赖项中删除了pyasn1和ndg-httpsclient。我们现在使用加密和idna,这两者都已经是PyOpenSSL的依赖。 (PR#930)

工作代码:

#!./venv/bin/python

import urllib3.contrib.pyopenssl
from argparse import ArgumentParser
import tdclient
import certifi
import urllib3

def main():
    urllib3.contrib.pyopenssl.inject_into_urllib3()
    key = getKey()
    listTdTables(key)
    http = urllib3.PoolManager(cert_reqs='CERT_REQUIRED', ca_certs=certifi.where())
    print(http.request('GET', 'https://www.google.com', timeout=4.0))
    print(http.request('GET', 'https://www.facebook.com', timeout=4.0))

def getKey():
    parser = ArgumentParser()
    parser.add_argument("-k", "--key", dest="key", help="key", required=True)
    options = parser.parse_args()
    return options.key

def listTdTables(key):
    with tdclient.Client(key) as client:
            for db in client.databases():
                    for table in db.tables():
                            print(table.db_name)
                            print(table.table_name)
                            print(table.count)

if __name__ == "__main__":
    main()