downloadPage回调函数问题

时间:2014-04-28 09:39:03

标签: python asynchronous twisted

我编写了以下python脚本:

from twisted.internet import defer             
from twisted.web.client import getPage, downloadPage, reactor
import tempfile

def success(results):
  print 'success'   

def error(results):
  print 'error', results
  reactor.stop()

tmpfilename = tempfile.mkstemp()
downloadPage('http://www.google.com', tmpfilename).addCallback(success).addErrback(error)

reactor.run()

我收到以下错误:

Unhandled Error
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/twisted/python/log.py", line 88, in callWithLogger
    return callWithContext({"system": lp}, func, *args, **kw)
  File "/usr/local/lib/python2.7/site-packages/twisted/python/log.py", line 73, in callWithContext
    return context.call({ILogContext: newCtx}, func, *args, **kw)
  File "/usr/local/lib/python2.7/site-packages/twisted/python/context.py", line 118, in callWithContext
    return self.currentContext().callWithContext(ctx, func, *args, **kw)
  File "/usr/local/lib/python2.7/site-packages/twisted/python/context.py", line 81, in callWithContext
    return func(*args,**kw)
--- <exception caught here> ---
  File "/usr/local/lib/python2.7/site-packages/twisted/internet/selectreactor.py", line 151, in _doReadOrWrite
    why = getattr(selectable, method)()
  File "/usr/local/lib/python2.7/site-packages/twisted/internet/tcp.py", line 215, in doRead
    return self._dataReceived(data)
  File "/usr/local/lib/python2.7/site-packages/twisted/internet/tcp.py", line 221, in _dataReceived
    rval = self.protocol.dataReceived(data)
  File "/usr/local/lib/python2.7/site-packages/twisted/protocols/basic.py", line 578, in dataReceived
    why = self.rawDataReceived(data)
  File "/usr/local/lib/python2.7/site-packages/twisted/web/http.py", line 518, in rawDataReceived
    self.handleResponsePart(data)
  File "/usr/local/lib/python2.7/site-packages/twisted/web/client.py", line 249, in handleResponsePart
    self.factory.pagePart(data)
  File "/usr/local/lib/python2.7/site-packages/twisted/web/client.py", line 504, in pagePart
    self.file.write(data)
exceptions.AttributeError: 'tuple' object has no attribute 'write'
Unhandled Error
Traceback (most recent call last):
  File "poc.py", line 16, in <module>
    reactor.run()
  File "/usr/local/lib/python2.7/site-packages/twisted/internet/base.py", line 1192, in run
    self.mainLoop()
  File "/usr/local/lib/python2.7/site-packages/twisted/internet/base.py", line 1204, in mainLoop
    self.doIteration(t)
  File "/usr/local/lib/python2.7/site-packages/twisted/internet/selectreactor.py", line 145, in doSelect
    _logrun(selectable, _drdw, selectable, method)
--- <exception caught here> ---
  File "/usr/local/lib/python2.7/site-packages/twisted/python/log.py", line 88, in callWithLogger
    return callWithContext({"system": lp}, func, *args, **kw)
  File "/usr/local/lib/python2.7/site-packages/twisted/python/log.py", line 73, in callWithContext
    return context.call({ILogContext: newCtx}, func, *args, **kw)
  File "/usr/local/lib/python2.7/site-packages/twisted/python/context.py", line 118, in callWithContext
    return self.currentContext().callWithContext(ctx, func, *args, **kw)
  File "/usr/local/lib/python2.7/site-packages/twisted/python/context.py", line 81, in callWithContext
    return func(*args,**kw)
  File "/usr/local/lib/python2.7/site-packages/twisted/internet/selectreactor.py", line 156, in _doReadOrWrite
    self._disconnectSelectable(selectable, why, method=="doRead")
  File "/usr/local/lib/python2.7/site-packages/twisted/internet/posixbase.py", line 263, in _disconnectSelectable
    selectable.connectionLost(failure.Failure(why))
  File "/usr/local/lib/python2.7/site-packages/twisted/internet/tcp.py", line 485, in connectionLost
    self._commonConnection.connectionLost(self, reason)
  File "/usr/local/lib/python2.7/site-packages/twisted/internet/tcp.py", line 299, in connectionLost
    protocol.connectionLost(reason)
  File "/usr/local/lib/python2.7/site-packages/twisted/web/client.py", line 198, in connectionLost
    http.HTTPClient.connectionLost(self, reason)
  File "/usr/local/lib/python2.7/site-packages/twisted/web/http.py", line 472, in connectionLost
    self.handleResponseEnd()
  File "/usr/local/lib/python2.7/site-packages/twisted/web/client.py", line 258, in handleResponseEnd
    self.factory.pageEnd()
  File "/usr/local/lib/python2.7/site-packages/twisted/web/client.py", line 531, in pageEnd
    self.file.close()
exceptions.AttributeError: 'tuple' object has no attribute 'close'

如果我将url更改为无效的内容,它将抛出正确的错误回调函数,因此它似乎与成功回调有关但是我无法理解为什么。

1 个答案:

答案 0 :(得分:3)

之后:

tmpfilename = tempfile.mkstemp()

tmpfilename的值是元组(请参阅docs),但twisted需要文件名或文件类对象。

所以你可以这样做:

tmpfile = tempfile.mkstemp()
tmpfilename = tmpfile[1]
downloadPage('http://www.google.com', tmpfilename).addCallback(success).addErrback(error)

有效。

但如果您不需要该文件继续存在,我建议您使用以下内容:

tmpfile = tempfile.TemporaryFile()
downloadPage('http://www.google.com', tmpfile).addCallback(success).addErrback(error)

使用TemporaryFile()构造函数,允许您访问下载的数据,但是一旦进程关闭,该文件(无论出于所有意图和目的)都无法再次被看到。

您可以使用上下文管理器进一步改进这一点 - 例如:

with tempfile.TemporaryFile() as tmpfile:
    downloadPage('http://www.google.com', tmpfile).addCallback(success).addErrback(error)

    # do other stuff with tmpfile

# code that no longer depends on the existence of tmpfile