龙卷风和aio_etcd建立锁定

时间:2016-11-05 08:31:08

标签: tornado etcd

我试图让Tornado使用aio_etcd aio_etcd documentation here

我希望获得Tornado Coroutine中的锁定。我写了以下代码。文档中的示例使用' await'然而,我用产量替换了这个,因为我使用@tornado.gen.coroutine装饰我不确定这是否正确。我通过以下代码得到以下崩溃:

    raise RuntimeError('Timeout context manager should be used '
RuntimeError: Timeout context manager should be used inside a task
ERROR:tornado.access:500 GET / (::1) 6.33ms

...

import tornado.web
import tornado.httpserver
import tornado.httpclient
import tornado.ioloop
import tornado.options
import tornado.gen
import tornado.auth
from multiprocessing import Process
import aio_etcd as etcd

def run_process(port):

   app=Application()
   server=tornado.httpserver.HTTPServer(app)
   server.listen(port)
   tornado.ioloop.IOLoop.current().start()

class  MainHandler(tornado.web.RequestHandler):
   @tornado.gen.coroutine
   def get(self):
      print("in function")

      l = etcd.Lock(self.application.etcdClient, "L")

      # Use the lock object:
      yield from l.acquire(blocking=True, lock_ttl=None)
      print("got lock")
      yield tornado.gen.sleep(30)
      yield from l.release()
      print("releasing lock")



class Application(tornado.web.Application):
   def __init__(self):
      handlers = [
         (r"/",MainHandler),
      ]

      self.etcdClient=etcd.Client()

      # Settings dict for Application
      settings = {
      }
      tornado.web.Application.__init__(self,handlers,debug=True,**settings)      

if __name__ =='__main__':
   Process(target=run_process,args=(8000,)).start()
   Process(target=run_process,args=(8001,)).start()

同样在我的代码中,我没有使用新的异步键盘而是我的所有Tornado协同例程都使用@tornado.gen.coroutine进行修饰。有人可以解释为什么使用yield关键字而不是yield in in in龙卷风的常规以及为什么这些不同。有谁知道我如何在Tornado中使用此代码?

使用asyncio更新代码:

import tornado.web
import tornado.httpserver
import tornado.httpclient
import tornado.ioloop
import tornado.options
import tornado.gen
import tornado.auth
import asyncio 
from multiprocessing import Process
import aio_etcd as etcd
import tornado.platform.asyncio 
def run_process(port):
   tornado.platform.asyncio.AsyncIOMainLoop().install()
   app=Application()
   server=tornado.httpserver.HTTPServer(app)
   server.listen(port)
   asyncio.get_event_loop().run_forever()

class  MainHandler(tornado.web.RequestHandler):
   async def get(self):

      print("here")
      lock = etcd.Lock(self.application.etcdClient, "hello")
      # Use the lock object:
      await asyncio.ensure_future(lock.acquire())
      state = await asyncio.ensure_future(lock.is_locked())
      print("lock state")
      print(state)

class Application(tornado.web.Application):
   def __init__(self):
      handlers = [
         (r"/",MainHandler),
      ]

      self.etcdClient=etcd.Client()

      # Settings dict for Application
      settings = {
      }
      tornado.web.Application.__init__(self,handlers,debug=True,**settings)      
if __name__ =='__main__':
   Process(target=run_process,args=(8000,)).start()
   Process(target=run_process,args=(8001,)).start()

崩溃追踪:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/tornado/web.py", line 1469, in _execute
    result = yield result
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/tornado/gen.py", line 1015, in run
    value = future.result()
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/tornado/concurrent.py", line 237, in result
    raise_exc_info(self._exc_info)
  File "<string>", line 3, in raise_exc_info
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/tornado/gen.py", line 1021, in run
    yielded = self.gen.throw(*exc_info)
  File "<string>", line 6, in _wrap_awaitable
  File "check3.py", line 27, in get
    await asyncio.ensure_future(lock.acquire())
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/asyncio/futures.py", line 361, in __iter__
    yield self  # This tells Task to wait for completion.
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/tornado/gen.py", line 1015, in run
    value = future.result()
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/tornado/concurrent.py", line 237, in result
    raise_exc_info(self._exc_info)
  File "<string>", line 3, in raise_exc_info
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/asyncio/tasks.py", line 239, in _step
    result = coro.send(None)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/aio_etcd-0.4.3.1-py3.5.egg/aio_etcd/lock.py", line 67, in acquire
    res = await self.client.write(self.path, self.uuid, ttl=lock_ttl, append=True)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/aio_etcd-0.4.3.1-py3.5.egg/aio_etcd/client.py", line 449, in write
    response = await self.api_execute(path, method, params=params)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/aio_etcd-0.4.3.1-py3.5.egg/aio_etcd/client.py", line 780, in wrapper
    response = await payload(self, path, method, params=params)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/aiohttp-1.1.1-py3.5-macosx-10.6-intel.egg/aiohttp/client.py", line 553, in __await__
    resp = yield from self._coro
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/aiohttp-1.1.1-py3.5-macosx-10.6-intel.egg/aiohttp/client.py", line 198, in _request
    proxy=proxy, proxy_auth=proxy_auth, timeout=timeout)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/aiohttp-1.1.1-py3.5-macosx-10.6-intel.egg/aiohttp/client_reqrep.py", line 79, in __init__
    url2 = url.with_query(params)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/yarl-0.5.3-py3.5-macosx-10.6-intel.egg/yarl/__init__.py", line 607, in with_query
    for k, v in query.items())
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/yarl-0.5.3-py3.5-macosx-10.6-intel.egg/yarl/__init__.py", line 607, in <genexpr>
    for k, v in query.items())
  File "yarl/_quoting.pyx", line 46, in yarl._quoting._quote (yarl/_quoting.c:1384)
TypeError: Argument should be str

2 个答案:

答案 0 :(得分:2)

正如Jesse所说,一些asyncio库特别需要asyncio,目前不能与Tornado一起使用(未来版本的Tornado将改善这种兼容性)。但是,有一种解决方法并不需要远离龙卷风。在屈服或等待之前,只需将aio_etcdasyncio.ensure_future()的任何电话联系起来。以下是aiohttp的示例(因为我没有etcd服务器进行测试):

from tornado.platform.asyncio import AsyncIOMainLoop
AsyncIOMainLoop().install()

from tornado.ioloop import IOLoop
import aiohttp
import asyncio

async def main():
    print(await asyncio.ensure_future(aiohttp.get('https://www.google.com')))

IOLoop.current().run_sync(main)

答案 1 :(得分:1)

为asyncio编写的许多库都可以与Tornado互操作。不幸的是,有些库,比如aio_etcd,不是,因为它们以特定于asyncio的方式实现超时。请参阅aiohttp issue 877 - 此问题描述了aiohttp.Timeout中的问题。该代码从aiohttp分离到单独的async_timeout包,并且这个单独的包仍然与Tornado不兼容。这是aio_etcd用来实现超时的,所以aio_etcd也与Tornado不兼容。

长期解决方法是在aio_etcd's tracker中打开一个要求Tornado兼容性的错误。如果timeout=None,aio_etcd可以使用async_timeout not 来提供Tornado兼容性。这与in this comment的建议相同。请在打开故障单时链接到此StackOverflow问题。

短期修复是使用asyncio及其Web框架aiohttp而不是Tornado:

from multiprocessing import Process

import asyncio
from aiohttp import web
import aio_etcd as etcd

# Don't create this in the parent process, wait for Process() to spawn child.
etcd_client = None

async def handle(request):
    print("in function")

    l = etcd.Lock(etcd_client, "L")

    # Use the lock object:
    await l.acquire(blocking=True, lock_ttl=None)
    print("got lock")
    await asyncio.sleep(30)
    await l.release()
    print("releasing lock")
    return web.Response(text='ok')


def run_process(port):
    global etcd_client
    app = web.Application(debug=True)
    app.router.add_get('/', handle)
    # Create AFTER multiprocess starts this child process.
    etcd_client = etcd.Client()
    web.run_app(app, port=port)


if __name__ == '__main__':
    # run_process(8000)
    Process(target=run_process, args=(8000,)).start()
    Process(target=run_process, args=(8001,)).start()

aiohttp docs to get you started are here

回答你的其他问题:龙卷风在gen.coroutine中使用了多年的“收益率”。它的设计影响了asyncio的协同程序,但它们在Python 3.4中使用了“yield from”。 Guido's explanation of the difference between "yield" and "yield from" is here,这当然很有趣,但现在不那么重要了。在Python 3.5中,“yield from”被PEP 492 -- Coroutines with async and await syntax取代。

如果您的代码永远不需要在早于3.5的Python上运行,只需使用“async”和“await”而不是“coroutine”装饰器和“yield”(在Tornado中)或“yield from”(在asyncio中)。对你而言,我知道你正在运行Python 3.5因为aio_etcd需要它,所以只需使用async并等待我在上面的代码中演示。