Question

在使用twistedmatrix ProxyClient时，我怎么可能使用gunzip和处理响应部分？

我需要检查文本或javascript和ajax查询/答案。我应该使用handleResponseEnd吗？

我认为它在handleResponsePart中，但看起来我误解了一个点或什么，这是我的骨架代码：

from twisted.python import log
from twisted.web import http, proxy

class ProxyClient(proxy.ProxyClient):
    """Mange returned header, content here.

    Use `self.father` methods to modify request directly.
    """
    def handleHeader(self, key, value):
            # change response header here
            log.msg("Header: %s: %s" % (key, value))
            proxy.ProxyClient.handleHeader(self, key, value)

    def handleResponsePart(self, buffer):
    # this part below do not work, 
    # looks like @ this moment i do not have 'Content-Encoding' or 'Content-Type'
    # what am i misunderstading?
            cEncoding = self.father.getAllHeaders().get('Content-Encoding', '')
            cType = self.father.getAllHeaders().get('Content-Type', '')
            print >> sys.stderr, 'Content-Encoding', cEncoding
            print >> sys.stderr, 'Content-Type', cType
            if ('text' in cType.lower() or 'javascript' in cType.lower()) and 'gzip' in cEncoding.lower():

                buf = StringIO(buffer)
                s = gzip.GzipFile(mode="rb", fileobj=buf)
                content = s.read(len(buffer))

                # here process content as it should be gunziped

    proxy.ProxyClient.handleResponsePart(self, buffer)

class ProxyClientFactory(proxy.ProxyClientFactory):
    protocol = ProxyClient

class ProxyRequest(proxy.ProxyRequest):
    protocols = dict(http=ProxyClientFactory)

class Proxy(proxy.Proxy):
    requestFactory = ProxyRequest

class ProxyFactory(http.HTTPFactory):
    protocol = Proxy

来自我的日志记录：

2013-06-11 14:07:33+0200 [ProxyClient,client] Header: Date: Tue, 11 Jun 2013 12:07:25 GMT
2013-06-11 14:07:33+0200 [ProxyClient,client] Header: Server: Apache
...
2013-06-11 14:07:33+0200 [ProxyClient,client] Header: Content-Type: text/html;charset=ISO-8859-1
...
2013-06-11 14:07:33+0200 [ProxyClient,client] Header: Content-Encoding: gzip
...
2013-06-11 14:07:33+0200 [ProxyClient,client] Header: Connection: close

因此我应该有两个条件好！我错过了什么？

即使我对第二种方式不感兴趣，也就是删除请求的接受，就像这样，是否可以这样做：（顺便说一下，它看起来不起作用，或者测试的网络服务器不关心我们不想接收gzip-ed内容的事实）

class ProxyRequest(proxy.ProxyRequest):
    protocols = dict(http=ProxyClientFactory)

    def process(self):
        # removing the accept so that we do not tell "i'm ok with gzip encoded content" and should receive only not gzip-ed
        self.requestHeaders.removeHeader('accept')
        self.requestHeaders.removeHeader('accept-encoding')

Answer 1

您必须在handleResponsePart中将数据块收集到StringIO缓冲区中，然后使用handleResponseEnd中的GzipFile进行解码。

python和twisted代理，如何在飞行中进行gunzip？

1 个答案: