Question

我有一个深度学习模型。为了查询模型，我使用celery使其异步，因为查询请求的数量非常大。启动服务时启动的工作人员数量等于系统中可用的核心数量。例如，如果我有一台8核计算机，它将最多增加8个工人。

我使用以下行使用celery进行异步请求：

response = flask_with_celery.inference.apply_async((service_id, locale, query), queue='inference')

其中inference是celery任务功能。 redis是celery的后端。

我使用jmeter进行性能测试。 x个并发用户的测试正在进行10分钟。我在这里使用x，20，30，40，50来尝试100的值。

一段时间后，在50个并发用户的末尾或100个并发用户的开始，代码将引发错误。

[2018-11-12 14:36:12,815] ERROR in app: Exception on /process_qna_query [POST]
   File "site-packages/flask/app.py", line 1982, in wsgi_app
   File "site-packages/flask/app.py", line 1614, in full_dispatch_request
   File "site-packages/flask_cors/extension.py", line 161, in wrapped_function
   File "site-packages/flask/app.py", line 1517, in handle_user_exception
   File "site-packages/flask/_compat.py", line 33, in reraise
   File "site-packages/flask/app.py", line 1612, in full_dispatch_request
   File "site-packages/flask/app.py", line 1598, in dispatch_request
   File "FAQ.py", line 103, in json_test
   File "site-packages/celery/result.py", line 194, in get
   File "site-packages/celery/backends/async.py", line 189, in wait_for_pending
   File "site-packages/celery/backends/async.py", line 256, in _wait_for_pending
   File "site-packages/celery/backends/async.py", line 57, in drain_events_until
   File "site-packages/celery/backends/async.py", line 66, in wait_for
   File "site-packages/celery/backends/redis.py", line 69, in drain_events
   File "site-packages/redis/client.py", line 2513, in get_message
   File "site-packages/redis/client.py", line 2428, in parse_response
   File "site-packages/redis/connection.py", line 619, in can_read
 ValueError: filedescriptor out of range in select()

如果我首先开始测试100个一致用户，然后是50，依此类推。与20个并发用户进行测试时出现错误。显然，这与用户数量无关。在100分钟内10个用户获得的请求数与20用户中获得的请求数大致相同。

编辑：我将芹菜结果后端更改为memcached，但没有收到任何错误消息！

ValueError：文件描述符超出select（）

0 个答案: