Question

我在redis中有一个任务队列，该作业以python异步模式运行。所以我在asyncio_redis中使用blpop。

async def do_job(job):
    print(f'doing job {job}')
    await asyncio.sleep(10)
    print('done!')


async def main():
    connection = await asyncio_redis.Connection.create(host='127.0.0.1', port=6379)
    while True:
        # may be 10000 jobs
        reply = await connection.blpop(['task-queue'])
        job_info = reply.value
        print(f'get job {job_info}')
        # I have to wait here
        await do_job(job_info)
        print(f'job\'s done!')


if __name__ == '__main__':
    asyncio.get_event_loop().run_until_complete(main())

问题出在redis blpop中，我不想等待工作完成，这太慢了。所以有没有办法像打击一样：

connection = await asyncio_redis.Connection.create(host='127.0.0.1', port=6379)
with ThreadPoolExecutor(max_workers=100) as executor:
    while True:
        reply = await connection.blpop(['task-queue'])
        job_info = reply.value
        # may block here when reach max_workers
        future = executor.submit(do_job, job_info)

就像ThreadPoolExecutor一样，但是asyncio版本！

我想我可以创建2个循环，1个用于redis循环，另一个用于工作。

但是我无法使它工作?

如何将作业提交到循环

0 个答案: