Question

我正在努力让网络刮刀与Scrapy合作。当我尝试刮取https网站时，我收到此错误：

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/twisted/web/client.py", line 1291, in request
    parsedURI.originForm)
  File "/usr/local/lib/python2.7/dist-packages/twisted/web/client.py", line 1182, in _requestWithEndpoint
    d = self._pool.getConnection(key, endpoint)
  File "/usr/local/lib/python2.7/dist-packages/twisted/web/client.py", line 1071, in getConnection
    return self._newConnection(key, endpoint)
  File "/usr/local/lib/python2.7/dist-packages/twisted/web/client.py", line 1083, in _newConnection
    return endpoint.connect(factory)
--- <exception caught here> ---
  File "/usr/local/lib/python2.7/dist-packages/twisted/internet/endpoints.py", line 869, in connect
    timeout=self._timeout, bindAddress=self._bindAddress)
  File "/usr/local/lib/python2.7/dist-packages/twisted/internet/posixbase.py", line 507, in connectSSL
    tlsFactory = tls.TLSMemoryBIOFactory(contextFactory, True, factory)
  File "/usr/local/lib/python2.7/dist-packages/twisted/protocols/tls.py", line 608, in __init__
    contextFactory.getContext()
  File "/usr/local/lib/python2.7/dist-packages/twisted/web/client.py", line 794, in getContext
    return self._webContext.getContext(self._hostname, self._port)
  File "/usr/local/lib/python2.7/dist-packages/scrapy/core/downloader/contextfactory.py", line 19, in getContext
    ctx.set_options(SSL.OP_ALL)
  File "build/bdist.linux-armv6l/egg/OpenSSL/SSL.py", line 753, in set_options

exceptions.OverflowError: integer 2147486719 does not fit '32-bit int'

这里似乎有一个错误修复，https://github.com/pyca/cryptography/issues/773但我有最新版本的加密（0.3）。

我能做什么的想法？

Answer 1

不幸的是，错误是误导性的。在Windows 32上，2147486719也不适合'long'（因为long也是4个字节）。

要解决此问题，请从https://github.com/pyca/cryptography下载最新的加密代码，然后在ssl.py中替换此行

long SSL_CTX_set_options(SSL_CTX *, long);

与

long SSL_CTX_set_options(SSL_CTX *, unsigned long);

重建＆amp;安装......事情应该可以正常工作。

python OpenSLL OverflowError

1 个答案: