Question

我的django芹菜任务只是部分执行。

我启动应用程序，连接看起来不错：

INFO / MainProcess]连接到 redis：//elasticache.cache.amazonaws.com：6379/0 [2018-02-17 23：27：24,314：INFO / MainProcess]混合：搜索邻居 [2018-02-17 23：27：25,339：INFO / MainProcess]混合：一个人 [2018-02-17 23：27：25,604：INFO / MainProcess] worker1 @ test_vmstracker_com ready。

我启动了这个过程，并且接收到了一个执行任务：

[2018-02-17 23：27：49,810：INFO / MainProcess]收到的任务： tracking.tasks.escalate [92f54d48202] ETA：[2018-02-18 07：27：59.797380 + 00:00 [2018-02-17 23：27：49,830：INFO / MainProcess] 收到的任务：tracking.tasks.escalate [09a0aebef72b] ETA：[2018-02-18 07：28：19.809712 + 00:00 [2018-02-17 23：28：00,205：警告/ ForkPoolWorker-7] -my app正在运行 -

然后我开始收到错误，但它没有完成我的应用发送电子邮件的任务

[2018-02-17 23：28：00,214：ERROR / ForkPoolWorker-7]与Redis的连接丢失：现在重试（0/20）。 [2018-02-17 23：28：00,220： ERROR / ForkPoolWorker-7]与Redis的连接丢失：重试（1/2

有谁知道为什么只执行，然后连接丢失？

这是完整的堆栈跟踪：

[2018-02-17 23:28:19,382: WARNING/ForkPoolWorker-7] /usr/local/lib/python3.6/site-packages/celery/app/trace.py:549: RuntimeWarning: Exception raised outside body: ConnectionError("Error while reading from socket: ('Connection closed by server.',)",):
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/redis/connection.py", line 177, in _read_from_socket
    raise socket.error(SERVER_CLOSED_CONNECTION_ERROR)
OSError: Connection closed by server.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/redis/client.py", line 2879, in execute
    return execute(conn, stack, raise_on_error)
  File "/usr/local/lib/python3.6/site-packages/redis/client.py", line 2764, in _execute_transaction
    self.parse_response(connection, '_')
  File "/usr/local/lib/python3.6/site-packages/redis/client.py", line 2838, in parse_response
    self, connection, command_name, **options)
  File "/usr/local/lib/python3.6/site-packages/redis/client.py", line 680, in parse_response
    response = connection.read_response()
  File "/usr/local/lib/python3.6/site-packages/redis/connection.py", line 624, in read_response
    response = self._parser.read_response()
  File "/usr/local/lib/python3.6/site-packages/redis/connection.py", line 284, in read_response
    response = self._buffer.readline()
  File "/usr/local/lib/python3.6/site-packages/redis/connection.py", line 216, in readline
    self._read_from_socket()
  File "/usr/local/lib/python3.6/site-packages/redis/connection.py", line 191, in _read_from_socket
    (e.args,))
redis.exceptions.ConnectionError: Error while reading from socket: ('Connection closed by server.',)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/redis/connection.py", line 177, in _read_from_socket
    raise socket.error(SERVER_CLOSED_CONNECTION_ERROR)
OSError: Connection closed by server.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/celery/app/trace.py", line 434, in trace_task
    uuid, retval, task_request, publish_result,
  File "/usr/local/lib/python3.6/site-packages/celery/backends/base.py", line 152, in mark_as_done
    self.store_result(task_id, result, state, request=request)
  File "/usr/local/lib/python3.6/site-packages/celery/backends/base.py", line 309, in store_result
    request=request, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/celery/backends/base.py", line 652, in _store_result
    self.set(self.get_key_for_task(task_id), self.encode(meta))
  File "/usr/local/lib/python3.6/site-packages/celery/backends/redis.py", line 213, in set
    return self.ensure(self._set, (key, value), **retry_policy)
  File "/usr/local/lib/python3.6/site-packages/celery/backends/redis.py", line 203, in ensure
    **retry_policy)
  File "/usr/local/lib/python3.6/site-packages/kombu/utils/functional.py", line 333, in retry_over_time
    return fun(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/celery/backends/redis.py", line 222, in _set
    pipe.execute()
  File "/usr/local/lib/python3.6/site-packages/redis/client.py", line 2894, in execute
    return execute(conn, stack, raise_on_error)
  File "/usr/local/lib/python3.6/site-packages/redis/client.py", line 2764, in _execute_transaction
    self.parse_response(connection, '_')
  File "/usr/local/lib/python3.6/site-packages/redis/client.py", line 2838, in parse_response
    self, connection, command_name, **options)
  File "/usr/local/lib/python3.6/site-packages/redis/client.py", line 680, in parse_response
    response = connection.read_response()
  File "/usr/local/lib/python3.6/site-packages/redis/connection.py", line 624, in read_response
    response = self._parser.read_response()
  File "/usr/local/lib/python3.6/site-packages/redis/connection.py", line 284, in read_response
    response = self._buffer.readline()
  File "/usr/local/lib/python3.6/site-packages/redis/connection.py", line 216, in readline
    self._read_from_socket()
  File "/usr/local/lib/python3.6/site-packages/redis/connection.py", line 191, in _read_from_socket
    (e.args,))
redis.exceptions.ConnectionError: Error while reading from socket: ('Connection closed by server.',)

  exc, exc_info.traceback)))

Answer 1

问题是我的app服务器和代理实例太小了。我使用的是ec2。一旦我升级到大型硬件，问题就消失了。 ec2实例或elasticache实例在CPU，内存和网络方面都太小了。

使用django芹菜在redis上丢失连接时执行的任务的一部分

1 个答案: