Question

在有关人脸识别的项目中，我将celery用作MQ。

我有三个任务队列“ task_gpu0，task_gpu1，task_download”，为gunicorn flask服务器提供服务，该服务器使用redis作为代理和后端。

当我使用jmeter对服务器施加压力时，大约20分钟后，程序会引发异常：

OperationalError：写入套接字时出现错误104。对等方重置连接。

在检查redis日志时，我发现info_clients中的client_recent_max_output_buffer和connected_clients继续增加。但是当我通过redis桌面管理器检查键值时，结果还可以。

我不知道为什么redis输出缓冲区和connected_clients继续增加。

redis日志：

:15:M 01 Apr 2019 09:11:18.113 # Client id=58081 addr=172.16.3.22:33832 fd=54 name= age=428 idle=0 flags=P db=1 sub=74995 psub=0 multi=-1 qbuf=79 qbuf-free=32689 obl=0 oll=452 omem=9267808 events=rw cmd=subscribe scheduled to be closed ASAP for overcoming of output buffer limits.

info clients:
connected_clients:587
client_recent_max_input_buffer:4
client_recent_max_output_buffer:55524832
blocked_clients:2

task_download.py

from celery import Celery
app = Celery()
app.config_from_object("celery_app_tmp.celeryconfig")

@app.task
def download(addImageInput, faceSetId):
    download_someting()
    return result_dict

task0.py与task1.py

相同

from celery import Celery
app = Celery()
app.config_from_object("celery_app_tmp.celeryconfig")

@app.task
def faceRec(addImageInput, faceSetId):
    do someting()
    return result_dict

celeryconfig.py

from kombu import Queue
from kombu import Exchange

result_serializer = 'msgpack'
task_serializer = 'msgpack'
accept_content = ['json', 'msgpack']

broker_url = "redis://:redis@172.16.3.22:7369/1"
result_backend = "redis://:redis@172.16.3.22:7369/1"

worker_concurrency = 8

result_exchange_type = 'direct'
result_expires = 5

task_queues = (
    Queue('gpu_0', exchange=Exchange('gpu_0'), routing_key='gpu_0'),
    Queue('gpu_1', exchange=Exchange('gpu_1'), routing_key='gpu_1'),

)

task_routes = {

                'celery_app_tmp.task0.faceRec': {'queue': 'gpu_0', 'routing_key': 'gpu_0'},
                'celery_app_tmp.task1.faceRec': {'queue': 'gpu_1', 'routing_key': 'gpu_1'},
                'celery_app_tmp.task_download.download': {'queue': 'download', 'routing_key': 'download'},
                'celery_app_tmp.task_download.delFile': {'queue': 'download', 'routing_key': 'download'}
}

main.py

import celery_app_tmp.task0
import celery_app_tmp.task1
import celery_app_tmp.task_download


result_exc = chain(celery_app_tmp.task_download.download.s(addImageInput), random.choice(list_task).faceRec.s(faceSetId))()
while True:
    if result_exc.ready():
        dict_exc = result_exc.get()
        break

Celery：client_recent_max_output_buffer和connected_clients的redis持续增加

0 个答案: