使用龙卷风和请求流式传输远程文件

时间:2014-10-23 01:29:54

标签: python tornado python-requests

基本上,我试图做的是打开与远程文件的连接并以块的形式读取然后将其拼写为用户

使用tornado作为Web框架并请求读取远程文件

这是我到目前为止的地方

class DefaultHandler(tornado.web.RequestHandler):
def get(self):
    url = 'http://domain.tld/large_file.rar'
    r = requests.get(url, stream=True)
    self.set_header('Content-Type', 'application/force-download')
    self.set_header('Content-Disposition', 'attachment; filename=large_file.rar')
    self.set_header('Content-length', r.headers.get('content-length'))
    self.set_header('Pragma', 'public')
    self.set_header('Expires', 0)
    self.set_header('Cache-Control', 'must-revalidate, post-check=0, pre-check=0')
    self.set_header('Cache-Controle', 'public')
    self.set_header('Content-Transfer-Encoding', 'binary')
    self.set_header('Accept-Ranges', 'bytes')
    self.set_header('Connection', 'close')
    for chunk in r.iter_content(chunk_size=1024):
        if chunk:
            self.write(chunk)
            self.flush()
    self.finish()

EDIT ------------------------------------------

class DefaultHandler(tornado.web.RequestHandler):
def get(self):

    url = 'http://largefilelocation'
    r = requests.Request('GET', url).prepare()
    resp = requests.Session().send(r, stream=True)

    self.set_header('Content-Type', 'application/force-download')
    self.set_header('Content-Disposition', resp.headers['content-disposition'])
    self.set_header('Pragma', 'public')
    self.set_header('Expires', 0)
    self.set_header('Content-Length', int(resp.headers['content-length']))
    self.set_header('Cache-Control', 'must-revalidate, post-check=0, pre-check=0')
    self.set_header('Cache-Controle', 'public')
    self.set_header('Content-Transfer-Encoding', 'binary')
    self.set_header('Accept-Ranges', 'bytes')
    self.set_header('Connection', 'close')

    for chunk in self._file_stream(resp):
        self.write(chunk)
        self.flush()
    self.finish()

def _file_stream(self, resp):
    print resp.headers
    for chunk in resp.iter_lines(128):
        if not chunk: break
        yield chunk

1 个答案:

答案 0 :(得分:3)

requests库是同步的,这意味着它会在您使用时阻止IOLoop。您必须使用Tornado的AsyncHTTPClient来允许IOLoop继续运行。 AsyncHTTPClient有一个streaming_callback选项,您可以在这里使用。