Python websockets,asyncio,queue - 自定义服务器类和带有生产者和消费者方法的处理程序类

时间:2016-11-21 14:55:51

标签: python websocket python-3.5 python-asyncio

我在理解asyncio工作流程时遇到了问题......

编辑 - 更改了代码以合并asyncio.Queue:

#!/usr/bin/env python

import asyncio
import websockets
import threading


class WSServer:

    def serve_forever(self):
        loop = asyncio.new_event_loop()
        asyncio.set_event_loop(loop)
        start_server = websockets.serve(self.handler, '127.0.0.1', 5678)
        asyncio.get_event_loop().run_until_complete(start_server)
        asyncio.get_event_loop().run_forever()

    async def handler(self, websocket, path):
        loop = asyncio.get_event_loop()
        master = MyClass(websocket)
        while True:
            listener_task = asyncio.ensure_future(master.get_message())
            producer_task = asyncio.ensure_future(master.produce())
            done, pending = await asyncio.wait(
                [listener_task, producer_task],
                return_when=asyncio.FIRST_COMPLETED)

            if listener_task in done:
                await master.consume()
            else:
                listener_task.cancel()

            if producer_task in done:
                msg_to_send = producer_task.result()
                await master.send_message(msg_to_send)
            else:
                producer_task.cancel()


class MyClass:

    incoming = asyncio.Queue()
    outgoing = asyncio.Queue()

    def __init__(self, websocket):
        self.ws = websocket

    async def get_message(self):
        msg_in = await self.ws.recv()
        await self.incoming.put(msg_in)

    async def send_message(self, message):
        await self.ws.send(message)

    async def consume(self):
        msg_to_consume = await self.incoming.get()
        # do something 'consuming' :)
        consume_output = msg_to_consume
        await self.outgoing.put(consume_output)

    async def produce(self):
        msg_out = await self.outgoing.get()
        return msg_out


if __name__ == '__main__':
    s = WSServer()
    t = threading.Thread(target=s.serve_forever)
    t.daemon = True
    t.start()
    while True:
        asyncio.sleep(5)

当改变MyClass.consume()时,它正在工作(在一台机器上,在另一台机器上,而不是lol),但有奇怪的行为:

async def consume(self):
    msg_to_consume = await self.incoming.get()
    # do something 'consuming' :)
    consume_output = msg_to_consume
    await self.outgoing.put('THIS WILL NOT GET INTO QUEUE???!!!')
    print('Outgoing empty 1: ' + str(self.outgoing.empty()))
    # And this will get into queue O.o
    await self.outgoing.put(consume_output)
    print('Outgoing empty 2: ' + str(self.outgoing.empty()))

我有两个等待,因为第一次调用self.outgoing.put()后,self.outgoing队列仍然是空的!只有当我再次打电话时,它似乎收到了这个项目......有什么想法吗?

其他机器只是抛出错误:

Exception in connection handler
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/websockets/server.py", line 78, in handler
    yield from self.ws_handler(self, path)
  File "test2.py", line 33, in handler
    msg_to_send = producer_task.result()
  File "/usr/lib/python3.5/asyncio/futures.py", line 274, in result
    raise self._exception
  File "/usr/lib/python3.5/asyncio/tasks.py", line 241, in _step
    result = coro.throw(exc)
  File "test2.py", line 66, in produce
    msg_out = await self.outgoing.get()
  File "/usr/lib/python3.5/asyncio/queues.py", line 168, in get
    yield from getter
  File "/usr/lib/python3.5/asyncio/futures.py", line 361, in __iter__
    yield self  # This tells Task to wait for completion.
RuntimeError: Task <Task pending coro=<MyClass.produce() running at test2.py:66> cb=[_wait.<locals>._on_completion() at /usr/lib/python3.5/asyncio/tasks.py:414]> got Future <Future pending> attached to a different loop

ORIGINAL:

我有这个代码,显然不能像我预期的那样工作:)

#!/usr/bin/env python

import asyncio
import websockets
import threading


class WSServer:

    def serve_forever(self):
        loop = asyncio.new_event_loop()
        asyncio.set_event_loop(loop)
        start_server = websockets.serve(self.handler, '127.0.0.1', 5678)
        asyncio.get_event_loop().run_until_complete(start_server)
        asyncio.get_event_loop().run_forever()

    async def handler(self, websocket, path):
        loop = asyncio.get_event_loop()
        master = MyClass(websocket)
        while True:
            listener_task = asyncio.ensure_future(master.get_message())
            producer_task = asyncio.ensure_future(master.produce())
            done, pending = await asyncio.wait(
                [listener_task, producer_task],
                return_when=asyncio.FIRST_COMPLETED)

            if listener_task in done:
                await master.consume()
            else:
                listener_task.cancel()

            if producer_task in done:
                if producer_task.result():
                    await master.send_message()
            else:
                producer_task.cancel()


class MyClass:

    incoming = []
    outgoing = []

    def __init__(self, websocket):
        self.ws = websocket

    async def get_message(self):
        self.incoming.append(self.ws.recv())

    async def send_message(self):
        self.ws.send(self.outgoing.pop(0))

    async def consume(self):
        self.outgoing.append(self.incoming.pop(0))

    async def produce(self):
        if self.outgoing:
            return True


if __name__ == '__main__':
    s = WSServer()
    t = threading.Thread(target=s.serve_forever)
    t.daemon = True
    t.start()
    while True:
        asyncio.sleep(5)

我想要实现的目标:

  1. 让WSServer实例在主线程的单独线程中运行(与WSServer.serve_forever配合使用)

  2. 对于连接的每个客户端,在WSServer.handler方法中创建包含两个列表的MyClass实例 - 一个用于传入消息,另一个用于传出。

  3. 传入应该从MyClass.get_message()填充 - 基本上是websocket.recv()

  4. 可以从MyClass.consume()填充传出 - 作为响应,但也可以从此代码范围之外填充。

  5. 当MyClass.incoming中存在某些内容时,通过MyClass.consume()处理它,当Myclass.outgoing中存在某些内容时,通过MyClass.send_message()进行处理

  6. 我不确定MyClass.produce(),因为我真的不需要生成任何东西,只有在有一些时才在传出中发送消息。我也看到了一些使用asycnio.Queue()

    的代码

    我在这里发现了类似的主题,但是他们的例子和问题是我的理解,说实话:

    Need help on producer and consumer thread in python

    asyncio queue consumer coroutine

    这里的正确方法应该是什么?

1 个答案:

答案 0 :(得分:1)

感谢python聊天室的帮助,找到了答案。

class MyClass:

    def __init__(self, websocket):
        self.ws = websocket
        self.incoming = asyncio.Queue()
        self.outgoing = asyncio.Queue()

应该为类的实例定义队列,而不是类本身。