如何制作GET请求,然后使用Python打印响应正文(扭曲)

时间:2015-10-05 20:41:56

标签: python http twisted twisted.client

在Twisted的网站上,他们有一个名为“接收响应”的示例,它向您展示如何获取响应标头和响应代码等,而不是响应主体(请求返回到网站的实际HTML主体)

在def cbrequest(响应)中,如何打印出来自example.com的GET请求返回的HTML文本?他们显示了像gets.headers这样的方法来获取标题,但我还没有看到一种返回前端主体的方法。

我尝试打印response.deliverBody(BeginningPrinter(已完成))以获取响应文本,但无济于事

http://twistedmatrix.com/documents/12.1.0/web/howto/client.html#auto9

from pprint import pformat

from twisted.internet import reactor
from twisted.internet.defer import Deferred
from twisted.internet.protocol import Protocol
from twisted.web.client import Agent
from twisted.web.http_headers import Headers

class BeginningPrinter(Protocol):
    def __init__(self, finished):
        self.finished = finished
        self.remaining = 1024 * 10

    def dataReceived(self, bytes):
        if self.remaining:
            display = bytes[:self.remaining]
            print 'Some data received:'
        print display
        self.remaining -= len(display)

 def connectionLost(self, reason):
    print 'Finished receiving body:', reason.getErrorMessage()
    self.finished.callback(None)

agent = Agent(reactor)
d = agent.request(
    'GET',
    'http://example.com/',
    Headers({'User-Agent': ['Twisted Web Client Example']}),
    None)

def cbRequest(response):
    print 'Response version:', response.version
    print 'Response code:', response.code
    print 'Response phrase:', response.phrase
    print 'Response headers:'
    print pformat(list(response.headers.getAllRawHeaders()))
    finished = Deferred()
    response.deliverBody(BeginningPrinter(finished))
    return finished
d.addCallback(cbRequest)

def cbShutdown(ignored):
   reactor.stop()
d.addBoth(cbShutdown)

reactor.run()

1 个答案:

答案 0 :(得分:1)

您必须保留数据并在之后将其返回

class BeginningPrinter(Protocol):
    def __init__(self, finished):
        self.finished = finished
        self.remaining = 1024 * 10
        self.buff = None

    def dataReceived(self, bytes):
        if self.buff is None:
            self.buff = bytes
        else:
            self.buff += bytes
        if self.remaining:
            display = bytes[:self.remaining]
            print 'Some data received:'
        print display
        self.remaining -= len(display)

    def connectionLost(self, reason):
        print 'Finished receiving body:', reason.getErrorMessage()
        self.finished.callback(self.buff)

请注意,您应该删除变量remaining,因为此代码示例要求服务器以1024 * 10个字节进行响应。

class BeginningPrinter(Protocol):
    def __init__(self, finished):
        self.finished = finished
        self.buff = None

    def dataReceived(self, bytes):
        if self.buff is None:
            self.buff = bytes
        else:
            self.buff += bytes

    def connectionLost(self, reason):
        print 'Finished receiving body:', reason.getErrorMessage()
        self.finished.callback(self.buff)