Question

我尝试使用twisted.protocols.tls实现一个可以通过TLS运行TLS的协议，startTLS是使用内存BIO的OpenSSL接口。

我将其实现为一个协议包装器，它大多看起来像常规的TCP传输，但它分别具有stopTLS和startTLS方法来添加和删除一层TLS。这适用于第一层TLS。如果我在“原生”Twisted TLS传输上运行它也可以正常工作。但是，如果我尝试使用此包装器提供的from twisted.python.components import proxyForInterface from twisted.internet.error import ConnectionDone from twisted.internet.interfaces import ITCPTransport, IProtocol from twisted.protocols.tls import TLSMemoryBIOFactory, TLSMemoryBIOProtocol from twisted.protocols.policies import ProtocolWrapper, WrappingFactory class TransportWithoutDisconnection(proxyForInterface(ITCPTransport)): """ A proxy for a normal transport that disables actually closing the connection. This is necessary so that when TLSMemoryBIOProtocol notices the SSL EOF it doesn't actually close the underlying connection. All methods except loseConnection are proxied directly to the real transport. """ def loseConnection(self): pass class ProtocolWithoutConnectionLost(proxyForInterface(IProtocol)): """ A proxy for a normal protocol which captures clean connection shutdown notification and sends it to the TLS stacking code instead of the protocol. When TLS is shutdown cleanly, this notification will arrive. Instead of telling the protocol that the entire connection is gone, the notification is used to unstack the TLS code in OnionProtocol and hidden from the wrapped protocol. Any other kind of connection shutdown (SSL handshake error, network hiccups, etc) are treated as real problems and propagated to the wrapped protocol. """ def connectionLost(self, reason): if reason.check(ConnectionDone): self.onion._stopped() else: super(ProtocolWithoutConnectionLost, self).connectionLost(reason) class OnionProtocol(ProtocolWrapper): """ OnionProtocol is both a transport and a protocol. As a protocol, it can run over any other ITransport. As a transport, it implements stackable TLS. That is, whatever application traffic is generated by the protocol running on top of OnionProtocol can be encapsulated in a TLS conversation. Or, that TLS conversation can be encapsulated in another TLS conversation. Or **that** TLS conversation can be encapsulated in yet *another* TLS conversation. Each layer of TLS can use different connection parameters, such as keys, ciphers, certificate requirements, etc. At the remote end of this connection, each has to be decrypted separately, starting at the outermost and working in. OnionProtocol can do this itself, of course, just as it can encrypt each layer starting with the innermost. """ def makeConnection(self, transport): self._tlsStack = [] ProtocolWrapper.makeConnection(self, transport) def startTLS(self, contextFactory, client, bytes=None): """ Add a layer of TLS, with SSL parameters defined by the given contextFactory. If *client* is True, this side of the connection will be an SSL client. Otherwise it will be an SSL server. If extra bytes which may be (or almost certainly are) part of the SSL handshake were received by the protocol running on top of OnionProtocol, they must be passed here as the **bytes** parameter. """ # First, create a wrapper around the application-level protocol # (wrappedProtocol) which can catch connectionLost and tell this OnionProtocol # about it. This is necessary to pop from _tlsStack when the outermost TLS # layer stops. connLost = ProtocolWithoutConnectionLost(self.wrappedProtocol) connLost.onion = self # Construct a new TLS layer, delivering events and application data to the # wrapper just created. tlsProtocol = TLSMemoryBIOProtocol(None, connLost, False) tlsProtocol.factory = TLSMemoryBIOFactory(contextFactory, client, None) # Push the previous transport and protocol onto the stack so they can be # retrieved when this new TLS layer stops. self._tlsStack.append((self.transport, self.wrappedProtocol)) # Create a transport for the new TLS layer to talk to. This is a passthrough # to the OnionProtocol's current transport, except for capturing loseConnection # to avoid really closing the underlying connection. transport = TransportWithoutDisconnection(self.transport) # Make the new TLS layer the current protocol and transport. self.wrappedProtocol = self.transport = tlsProtocol # And connect the new TLS layer to the previous outermost transport. self.transport.makeConnection(transport) # If the application accidentally got some bytes from the TLS handshake, deliver # them to the new TLS layer. if bytes is not None: self.wrappedProtocol.dataReceived(bytes) def stopTLS(self): """ Remove a layer of TLS. """ # Just tell the current TLS layer to shut down. When it has done so, we'll get # notification in *_stopped*. self.transport.loseConnection() def _stopped(self): # A TLS layer has completely shut down. Throw it away and move back to the # TLS layer it was wrapping (or possibly back to the original non-TLS # transport). self.transport, self.wrappedProtocol = self._tlsStack.pop()方法添加第二个TLS层，则会立即出现握手错误，并且连接会以某种未知的不可用状态结束。

包装器和让它工作的两个帮助器看起来像这样：

bzr branch lp:~exarkun/+junk/onion

我有简单的客户端和服务器程序来执行此操作，可从启动板（startTLS）获得。当我用它来调用上面的stopTLS方法两次，而没有对OpenSSL.SSL.Error: [('SSL routines', 'SSL23_GET_SERVER_HELLO', 'unknown protocol')]进行干预时，会出现这个OpenSSL错误：

{{1}}

为什么出问题？

Answer 1

OnionProtocol至少存在两个问题：

最里面 TLSMemoryBIOProtocol成为wrappedProtocol，当它应该是最外层;
ProtocolWithoutConnectionLost不会弹出任何TLSMemoryBIOProtocol关闭OnionProtocol的堆栈，因为connectionLost仅在FileDescriptor s {doRead之后调用1}}或doWrite方法返回断开连接的原因。

我们无法在不改变OnionProtocol管理其堆栈的方式的情况下解决第一个问题，在找出新的堆栈实现之前，我们无法解决第二个问题。不出所料，正确的设计是数据在Twisted中流动的直接结果，因此我们将从一些数据流分析开始。

Twisted表示与twisted.internet.tcp.Server或twisted.internet.tcp.Client的实例建立的连接。由于我们程序中唯一的交互性发生在stoptls_client，因此我们只考虑进出Client实例的数据流。

让我们用最小的LineReceiver客户端进行预热，该客户端回送从端口9999上的本地服务器接收的线路：

from twisted.protocols import basic
from twisted.internet import defer, endpoints, protocol, task

class LineReceiver(basic.LineReceiver):
    def lineReceived(self, line):
        self.sendLine(line)

def main(reactor):
    clientEndpoint = endpoints.clientFromString(
        reactor, "tcp:localhost:9999")
    connected = clientEndpoint.connect(
        protocol.ClientFactory.forProtocol(LineReceiver))
    def waitForever(_):
        return defer.Deferred()
    return connected.addCallback(waitForever)

task.react(main)

建立连接后，Client成为我们LineReceiver协议的传输并调解输入和输出：

来自服务器的新数据导致反应堆调用Client的{{1}}方法，后者又将其收到的内容传递给doRead＆＃ 39; s LineReceiver方法。最后，dataReceived在至少有一行可用时调用LineReceiver.dataReceived。

我们的应用程序通过调用LineReceiver.lineReceived将一行数据发送回服务器。这会在绑定到协议实例的传输上调用LineReceiver.sendLine，这是处理传入数据的write实例。 Client安排数据由反应堆发送，而Client.write实际上通过套接字发送数据。

我们已准备好查看从未调用Client.doWrite的{{3}}的行为：

OnionClient

startTLS包含在中，这是我们尝试嵌套TLS的关键。作为OnionProtocols的子类，OnionClient的实例是一种协议传输三明治;它将自身表示为较低级别传输的协议，并将其作为传输呈现给协议，它通过twisted.internet.policies.ProtocolWrapper在连接时建立的伪装进行包装。

现在，OnionProtocol调用Client.doRead，将数据代理到OnionProtocol.dataReceived。作为OnionClient的传输，OnionClient接受从OnionProtocol.write发送的行，并将其代理为OnionClient.sendLine，其拥有传输。这是Client，它的包装协议和它自己的传输之间的正常交互，因此数据自然地流入和流出每个数据而没有任何麻烦。

ProtocolWrapper做了不同的事情。它试图在建立的协议传输对之间插入一个新的OnionProtocol.startTLS - 恰好是WrappingFactory。这看起来很简单：ProtocolWrapper将上层协议存储为TLSMemoryBIOProtocol和wrappedProtocol attribute。 ProtocolWrapper应该能够通过在自己的startTLS和TLSMemoryBIOProtocol上修补该实例，将OnionClient新的wrappedProtocol注入到连接中：

transport

这是第一次调用def startTLS(self): ... connLost = ProtocolWithoutConnectionLost(self.wrappedProtocol) connLost.onion = self # Construct a new TLS layer, delivering events and application data to the # wrapper just created. tlsProtocol = TLSMemoryBIOProtocol(None, connLost, False) # Push the previous transport and protocol onto the stack so they can be # retrieved when this new TLS layer stops. self._tlsStack.append((self.transport, self.wrappedProtocol)) ... # Make the new TLS layer the current protocol and transport. self.wrappedProtocol = self.transport = tlsProtocol后的数据流：

proxies write and other attributes down to its own transport

正如预期的那样，发送到startTLS的新数据会被路由到OnionProtocol.dataReceived上存储的TLSMemoryBIOProtocol，_tlsStack将解密的明文传递给OnionClient.dataReceived。 OnionClient.sendLine也会将其数据传递给TLSMemoryBIOProtocol.write，OnionProtocol.write对其进行加密并将生成的密文发送至Client.write，然后startTLS。

不幸的是，这个方案在第二次调用self.wrappedProtocol = self.transport = tlsProtocol后失败了。根本原因是这一行：

startTLS

对wrappedProtocol的每次调用都会将TLSMemoryBIOProtocol替换为最里面的 Client.doRead，即使transport收到的数据已由<{1}}加密EM>最外层的：

然而，OnionClient.sendLine是正确嵌套的。 write只能调用其传输OnionProtocol.write - 即OnionProtocol - 所以transport应将TLSMemoryBIOProtocol替换为最内层的TLSMemoryBIOProtocol _tlsStack确保写入连续嵌套在其他加密层中。

然后，解决方案是确保数据依次通过_tlsStack上的 wrappedProtocol流向 next ，以便按照应用的相反顺序剥离每层加密：

根据这一新要求，将ProtocolWrapper表示为列表似乎不太自然。幸运的是，线性表示传入的数据流表明了一种新的数据结构：

传入数据的错误和正确流量类似于单链接列表，protocol用作Client的下一个链接，OnionProtocol用作OnionClient＆＃ 39; S。该列表应从transport向下增长，并始终以transport.write结尾。发生该错误是因为违反了排序不变量。

单链表可以很好地将协议推送到堆栈上但是很难将它们弹出，因为它需要从头部向下遍历到要删除的节点。当然，每次收到数据时都会发生这种遍历，因此关注的是额外遍历所隐含的复杂性，而不是最坏情况下的时间复杂度。幸运的是，该列表实际上是双重关联的：

Client属性将每个嵌套协议与其前任协议相链接，以便OnionClient可以在最终通过网络发送数据之前连续降低加密级别。我们有两个哨兵来帮助管理列表：from twisted.python.components import proxyForInterface from twisted.internet.interfaces import ITCPTransport from twisted.protocols.tls import TLSMemoryBIOFactory, TLSMemoryBIOProtocol from twisted.protocols.policies import ProtocolWrapper, WrappingFactory class PopOnDisconnectTransport(proxyForInterface(ITCPTransport)): """ L{TLSMemoryBIOProtocol.loseConnection} shuts down the TLS session and calls its own transport's C{loseConnection}. A zero-length read also calls the transport's C{loseConnection}. This proxy uses that behavior to invoke a C{pop} callback when a session has ended. The callback is invoked exactly once because C{loseConnection} must be idempotent. """ def __init__(self, pop, **kwargs): super(PopOnDisconnectTransport, self).__init__(**kwargs) self._pop = pop def loseConnection(self): self._pop() self._pop = lambda: None class OnionProtocol(ProtocolWrapper): """ OnionProtocol is both a transport and a protocol. As a protocol, it can run over any other ITransport. As a transport, it implements stackable TLS. That is, whatever application traffic is generated by the protocol running on top of OnionProtocol can be encapsulated in a TLS conversation. Or, that TLS conversation can be encapsulated in another TLS conversation. Or **that** TLS conversation can be encapsulated in yet *another* TLS conversation. Each layer of TLS can use different connection parameters, such as keys, ciphers, certificate requirements, etc. At the remote end of this connection, each has to be decrypted separately, starting at the outermost and working in. OnionProtocol can do this itself, of course, just as it can encrypt each layer starting with the innermost. """ def __init__(self, *args, **kwargs): ProtocolWrapper.__init__(self, *args, **kwargs) # The application level protocol is the sentinel at the tail # of the linked list stack of protocol wrappers. The stack # begins at this sentinel. self._tailProtocol = self._currentProtocol = self.wrappedProtocol def startTLS(self, contextFactory, client, bytes=None): """ Add a layer of TLS, with SSL parameters defined by the given contextFactory. If *client* is True, this side of the connection will be an SSL client. Otherwise it will be an SSL server. If extra bytes which may be (or almost certainly are) part of the SSL handshake were received by the protocol running on top of OnionProtocol, they must be passed here as the **bytes** parameter. """ # The newest TLS session is spliced in between the previous # and the application protocol at the tail end of the list. tlsProtocol = TLSMemoryBIOProtocol(None, self._tailProtocol, False) tlsProtocol.factory = TLSMemoryBIOFactory(contextFactory, client, None) if self._currentProtocol is self._tailProtocol: # This is the first and thus outermost TLS session. The # transport is the immutable sentinel that no startTLS or # stopTLS call will move within the linked list stack. # The wrappedProtocol will remain this outermost session # until it's terminated. self.wrappedProtocol = tlsProtocol nextTransport = PopOnDisconnectTransport( original=self.transport, pop=self._pop ) # Store the proxied transport as the list's head sentinel # to enable an easy identity check in _pop. self._headTransport = nextTransport else: # This a later TLS session within the stack. The previous # TLS session becomes its transport. nextTransport = PopOnDisconnectTransport( original=self._currentProtocol, pop=self._pop ) # Splice the new TLS session into the linked list stack. # wrappedProtocol serves as the link, so the protocol at the # current position takes our new TLS session as its # wrappedProtocol. self._currentProtocol.wrappedProtocol = tlsProtocol # Move down one position in the linked list. self._currentProtocol = tlsProtocol # Expose the new, innermost TLS session as the transport to # the application protocol. self.transport = self._currentProtocol # Connect the new TLS session to the previous transport. The # transport attribute also serves as the previous link. tlsProtocol.makeConnection(nextTransport) # Left over bytes are part of the latest handshake. Pass them # on to the innermost TLS session. if bytes is not None: tlsProtocol.dataReceived(bytes) def stopTLS(self): self.transport.loseConnection() def _pop(self): pop = self._currentProtocol previous = pop.transport # If the previous link is the head sentinel, we've run out of # linked list. Ensure that the application protocol, stored # as the tail sentinel, becomes the wrappedProtocol, and the # head sentinel, which is the underlying transport, becomes # the transport. if previous is self._headTransport: self._currentProtocol = self.wrappedProtocol = self._tailProtocol self.transport = previous else: # Splice out a protocol from the linked list stack. The # previous transport is a PopOnDisconnectTransport proxy, # so first retrieve proxied object off its original # attribute. previousProtocol = previous.original # The previous protocol's next link becomes the popped # protocol's next link previousProtocol.wrappedProtocol = pop.wrappedProtocol # Move up one position in the linked list. self._currentProtocol = previousProtocol # Expose the new, innermost TLS session as the transport # to the application protocol. self.transport = self._currentProtocol class OnionFactory(WrappingFactory): """ A L{WrappingFactory} that overrides L{WrappingFactory.registerProtocol} and L{WrappingFactory.unregisterProtocol}. These methods store in and remove from a dictionary L{ProtocolWrapper} instances. The C{transport} patching done as part of the linked-list management above causes the instances' hash to change, because the C{__hash__} is proxied through to the wrapped transport. They're not essential to this program, so the easiest solution is to make them do nothing. """ protocol = OnionProtocol def registerProtocol(self, protocol): pass def unregisterProtocol(self, protocol): pass必须始终位于顶部，PopOnDisconnectTransport必须始终位于底部。

将两者放在一起，我们最终得到了这个：

connectionLost

（这也可在上找到。）

第二个问题的解决方案在于connectionLost。原始代码尝试通过TLSMemoryBIOProtocol从堆栈中弹出TLS会话，但由于只有一个已关闭的文件描述符导致loseConnection被调用，因此无法删除已停止的TLS会话关闭底层套接字。

在撰写本文时，_shutdownTLS仅在两个地方调用其传输_tlsShutdownFinished：GitHub和_shutdownTLS。在主动关闭时调用loseConnection _tlsShutdownFinished，loseConnection，abortConnection和unregisterProducer），而在被动关闭时调用PopOnDisconnectTransport loseConnection 3}}，after loseConnection and all pending writes have been flushed，handshake failures和empty reads）。这意味着，在TLSMemoryBIOProtocol期间，关闭连接的两个侧可以在堆栈中弹出已停止的TLS会话。 loseConnection这意味着这是因为TLSMemoryBIOProtocol通常是幂等的，而{{1}}肯定是期望的。

将堆栈管理逻辑放在{{1}}中的缺点是它取决于{{1}}实现的细节。通用解决方案需要跨越多个Twisted级别的新API。

在那之前，我们仍然坚持另一个read errors的例子。

Answer 2

如果该设备具有此功能，您可能需要通知远程设备您希望启动环境并为启动它之前为第二层分配资源。

Answer 3

如果您对两个图层使用相同的TLS参数并且要连接到同一主机，那么您可能对两个加密层使用相同的密钥对。尝试为嵌套层使用不同的密钥对，例如隧道连接到第三个主机/端口。即：localhost:30000（客户） - ＆gt; localhost:8080（使用密钥对A的TLS第1层） - ＆gt; localhost:8081（使用密钥对B的TLS第2层）。

尝试使用此代码在TLS上运行TLS时，为什么会出现握手失败？

3 个答案: