Python tornado异步请求处理程序

时间:2015-10-22 19:52:49

标签: python asynchronous tornado

我写了一个异步文件上传RequestHandler。它是按字节顺序正确的,即我收到的文件与发送的文件相同。我无法搞清楚的一个问题是上传延迟。特别是当我在本地测试时发出上传文件的post请求时,我看到显示上传进度的浏览器卡住了。对于接近4MB大小的文件,它会在50%+上停留一段时间,然后一段时间过去并发送所有数据,然后卡在"等待localhost ..."整个过程可能持续3分钟以上。

当我添加以data_received方法的新行结尾的print语句时,延迟消失了。 print语句是否会以某种方式触发网络缓冲区?

这是data_received的实现,以及辅助方法:

@tornado.gen.coroutine
def _read_data(self, cont_buf):
    '''
    Read the file data.

    @param cont_buf - buffered HTTP request
    @param boolean indicating whether data is still being read and new
           buffer
    '''
    # Check only last characters of the buffer guaranteed to be large
    # enough to contain the boundary
    end_of_data_idx = cont_buf.find(self._boundary)

    if end_of_data_idx >= 0:
        data = cont_buf[:(end_of_data_idx - self.LSEP)]
        self.receive_data(self.header_list[-1], data)
        new_buffer = cont_buf[(end_of_data_idx + len(self._boundary)):]
        return False, new_buffer
    else:
        self.receive_data(self.header_list[-1], cont_buf)
        return True, b""

@tornado.gen.coroutine
def _parse_params(self, param_buf):
    '''
    Parse HTTP header parameters.

    @param param_buf - string buffer containing the parameters.
    @returns parameters dictionary
    '''
    params = dict()
    param_res = self.PAT_HEADERPARAMS.findall(param_buf)
    if param_res:
        for name, value in param_res:
            params[name] = value
    elif param_buf:
        params['value'] = param_buf
    return params

@tornado.gen.coroutine
def _parse_header(self, header_buf):
    '''
    Parses a buffer containing an individual header with parameters.

    @param header_buf - header buffer containing a single header
    @returns header dictionary
    '''
    res = self.PAT_HEADERVALUE.match(header_buf)
    header = dict()
    if res:
        name, value, tail = res.groups()
        header = {'name': name, 'value': value,
                  'params': (yield self._parse_params(tail))}
    elif header_buf:
        header = {"value": header_buf}
    return header

@tornado.gen.coroutine
def data_received(self, chunk):
    '''
    Processes a chunk of content body.

    @param chunk - a piece of content body.
    '''
    self._count += len(chunk)
    self._buffer += chunk
    # Has boundary been established?
    if not self._boundary:
        self._boundary, self._buffer =\
            (yield self._extract_boundary(self._buffer))

        if (not self._boundary
                and len(self._buffer) > self.BOUNDARY_SEARCH_BUF_SIZE):
            raise RuntimeError("Cannot find multipart delimiter.")

    while True:
        if self._receiving_data:
            self._receiving_data, self._buffer = yield self._read_data(self._buffer)
            if self._is_end_of_request(self._buffer):
                yield self.request_done()
                break
            elif self._is_end_of_data(self._buffer):
                break
        else:
            headers, self._buffer = yield self._read_headers(self._buffer)
            if headers:
                self.header_list.append(headers)
                self._receiving_data = True
            else:
                break

0 个答案:

没有答案