Question

我使用tornado和asyncmc来构建异步API服务。它运行良好几周，但今天问题来了。几乎所有由asyncmc缓存的服务都会引发StreamClosedError。

从龙卷风日志中，它说：

ile "/opt/python2.7/lib/python2.7/site-packages/tornado-4.1-py2.7-linux-x86_64.egg/tornado/web.py", line 1348, in _execute
    result = yield result
  File "/opt/python2.7/lib/python2.7/site-packages/tornado-4.1-py2.7-linux-x86_64.egg/tornado/gen.py", line 807, in run
    value = future.result()
  File "/opt/python2.7/lib/python2.7/site-packages/tornado-4.1-py2.7-linux-x86_64.egg/tornado/concurrent.py", line 209, in result
    raise_exc_info(self._exc_info)
  File "/opt/python2.7/lib/python2.7/site-packages/tornado-4.1-py2.7-linux-x86_64.egg/tornado/gen.py", line 810, in run
    yielded = self.gen.throw(*sys.exc_info())
  File "/opt/app/python/tv-appstore/handler/base.py", line 100, in wrapper
    result = yield mc.get(self.cache_key)
  File "/opt/python2.7/lib/python2.7/site-packages/tornado-4.1-py2.7-linux-x86_64.egg/tornado/gen.py", line 807, in run
    value = future.result()
  File "/opt/python2.7/lib/python2.7/site-packages/tornado-4.1-py2.7-linux-x86_64.egg/tornado/concurrent.py", line 209, in result
    raise_exc_info(self._exc_info)
  File "/opt/python2.7/lib/python2.7/site-packages/tornado-4.1-py2.7-linux-x86_64.egg/tornado/gen.py", line 810, in run
    yielded = self.gen.throw(*sys.exc_info())
  File "build/bdist.linux-x86_64/egg/asyncmc/client.py", line 63, in wrapper
    res = yield func(self, conn, *args, **kwargs)
  File "/opt/python2.7/lib/python2.7/site-packages/tornado-4.1-py2.7-linux-x86_64.egg/tornado/gen.py", line 807, in run
    value = future.result()
  File "/opt/python2.7/lib/python2.7/site-packages/tornado-4.1-py2.7-linux-x86_64.egg/tornado/concurrent.py", line 209, in result
    raise_exc_info(self._exc_info)
  File "/opt/python2.7/lib/python2.7/site-packages/tornado-4.1-py2.7-linux-x86_64.egg/tornado/gen.py", line 810, in run
    yielded = self.gen.throw(*sys.exc_info())
  File "build/bdist.linux-x86_64/egg/asyncmc/client.py", line 238, in get
    result = yield self._multi_get(conn, self._key_type(key=key))
  File "/opt/python2.7/lib/python2.7/site-packages/tornado-4.1-py2.7-linux-x86_64.egg/tornado/gen.py", line 807, in run
    value = future.result()
  File "/opt/python2.7/lib/python2.7/site-packages/tornado-4.1-py2.7-linux-x86_64.egg/tornado/concurrent.py", line 209, in result
    raise_exc_info(self._exc_info)
  File "/opt/python2.7/lib/python2.7/site-packages/tornado-4.1-py2.7-linux-x86_64.egg/tornado/gen.py", line 810, in run
    yielded = self.gen.throw(*sys.exc_info())
  File "build/bdist.linux-x86_64/egg/asyncmc/client.py", line 278, in _multi_get
    servers_resp = yield conn.send_cmd_all(cmd, stream=True)
  File "/opt/python2.7/lib/python2.7/site-packages/tornado-4.1-py2.7-linux-x86_64.egg/tornado/gen.py", line 807, in run
    value = future.result()
  File "/opt/python2.7/lib/python2.7/site-packages/tornado-4.1-py2.7-linux-x86_64.egg/tornado/concurrent.py", line 209, in result
    raise_exc_info(self._exc_info)
  File "/opt/python2.7/lib/python2.7/site-packages/tornado-4.1-py2.7-linux-x86_64.egg/tornado/gen.py", line 810, in run
    yielded = self.gen.throw(*sys.exc_info())
  File "build/bdist.linux-x86_64/egg/asyncmc/pool.py", line 91, in send_cmd_all
    server_resp = yield host.send_cmd(cmd, *arg, **kw)
  File "/opt/python2.7/lib/python2.7/site-packages/tornado-4.1-py2.7-linux-x86_64.egg/tornado/gen.py", line 807, in run
    value = future.result()
  File "/opt/python2.7/lib/python2.7/site-packages/tornado-4.1-py2.7-linux-x86_64.egg/tornado/concurrent.py", line 209, in result
    raise_exc_info(self._exc_info)
  File "/opt/python2.7/lib/python2.7/site-packages/tornado-4.1-py2.7-linux-x86_64.egg/tornado/gen.py", line 212, in wrapper
    yielded = next(result)
  File "build/bdist.linux-x86_64/egg/asyncmc/host.py", line 77, in send_cmd
    yield self.stream.write(cmd)
  File "/opt/python2.7/lib/python2.7/site-packages/tornado-4.1-py2.7-linux-x86_64.egg/tornado/iostream.py", line 356, in write
    self._check_closed()
  File "/opt/python2.7/lib/python2.7/site-packages/tornado-4.1-py2.7-linux-x86_64.egg/tornado/iostream.py", line 864, in _check_closed
    raise StreamClosedError("Stream is closed")

Answer 1

我有同样的问题。当您将minsize（对于asyncmc.Client）设置为大于1并且某些连接到ConnectionPool的连接因使用库（tornado，asyncmc）之外的原因而被破坏时，可能会发生这种情况。为了解决这个问题，我通过自己的方法从缓存中获取值：

self.mc_get(key)

而不是

self.mc.get(key)

并调用 self.mc.pool.clear（） 来清理连接池并初始＆＃34;完全重新连接＆＃34;。

Python代码：

import tornado
from tornado.iostream import StreamClosedError

class SomeBaseHandler(tornado.web.RequestHandler):

    def mc_get(self, key, default=None):
        """ Additional interlayer for debug and logging requested keys from memcached

        Args:
            key (str): name of key in memcached
            default: default value, returns if key not exists

        Returns:
            obj
        """
        try:
            return yield self.mc.get(key, default)
        except StreamClosedError as e:
            self.mc.pool.clear()
            # Here you can raise own exception
            return default

    # …

我没有使用递归调用mc_get来避免无限循环，而无需在方法中添加额外的代码。

Tornado引发StreamClosedError：Stream关闭

1 个答案: