Question

我的问题是当我使用芹菜任务启动我的工作时，redis在终端显示以下错误。

我很幸运能够在celery documentation中使用修改后的和弦版本重现错误。似乎问题出现在和弦或我有太多组并行运行的地方，chord(add.s(i, i) for i in range(1, num))(list_add.s())或group(add.s(i, i) for i in range(1, num))()

以下是我的代码示例

@task
def add(x, y):
    return [x, y, x + y, "Next"]

@task
def list_add(nums):
    numbers = []
    count = 1
    for i in nums:
        print("{}). {}".format(count, i))
        numbers.extend(i)
        count += 1
    print(numbers)
    return numbers

@task
def foo(num):
    return chord(add.s(i, i) for i in range(1, num))(list_add.s())

以下是我的终端输出的部分回溯。

    [2015-11-04 20:36:14,912: INFO/MainProcess] Received task: b2b.tasks.add[b87fdc44-e759-4224-bce4-11f9468d12b3]
[2015-11-04 20:36:14,913: INFO/MainProcess] Received task: b2b.tasks.add[120f5bf2-b962-4424-894b-d6f0ca56102b]
[2015-11-04 20:36:14,914: INFO/MainProcess] Task b2b.tasks.bar[9dc93c75-6404-4db3-a685-ff91460e1adb] succeeded in 1.00830382s: <AsyncResult: c891df3e-aa5c-4c9f-ad3f-30abe3b3ccc1>

    [2015-11-04 19:44:34,922: ERROR/MainProcess] Connection to Redis lost: Retry (0/20) now.
    [2015-11-04 19:44:34,922: ERROR/MainProcess] Connection to Redis lost: Retry (1/20) in 1.00 second.
    ...
    /Users/Me/.virtualenvs/djangoscrape/lib/python2.7/site-packages/celery/app/trace.py:365: RuntimeWarning: Exception raised outside body: ConnectionError('Error 8 connecting to localhost:6379. nodename nor servname provided, or not known.',):
    Traceback (most recent call last):
      File "/Users/Me/.virtualenvs/djangoscrape/lib/python2.7/site-packages/celery/app/trace.py", line 235, in trace_task
      File "/Users/Me/.virtualenvs/djangoscrape/lib/python2.7/site-packages/celery/backends/base.py", line 256, in store_result
      File "/Users/Me/.virtualenvs/djangoscrape/lib/python2.7/site-packages/celery/backends/base.py", line 490, in _store_result
      File "/Users/Me/.virtualenvs/djangoscrape/lib/python2.7/site-packages/celery/backends/redis.py", line 160, in set
      File "/Users/Me/.virtualenvs/djangoscrape/lib/python2.7/site-packages/celery/backends/redis.py", line 149, in ensure
      File "/Users/Me/.virtualenvs/djangoscrape/lib/python2.7/site-packages/kombu/utils/__init__.py", line 243, in retry_over_time
      File "/Users/Me/.virtualenvs/djangoscrape/lib/python2.7/site-packages/celery/backends/redis.py", line 169, in _set
      File "/Users/Me/.virtualenvs/djangoscrape/lib/python2.7/site-packages/redis/client.py", line 2593, in execute
      File "/Users/Me/.virtualenvs/djangoscrape/lib/python2.7/site-packages/redis/client.py", line 2447, in _execute_transaction
      File "/Users/Me/.virtualenvs/djangoscrape/lib/python2.7/site-packages/redis/connection.py", line 532, in send_packed_command
      File "/Users/Michael/.virtualenvs/djangoscrape/lib/python2.7/site-packages/redis/connection.py", line 436, in connect
    ConnectionError: Error 8 connecting to localhost:6379. nodename nor servname provided, or not known.

这是我用来在终端启动我的工作人员的命令芹菜 - 刮刀工人-P eventlet -c 1000 -l info

当foo（）传递20但是1000或更大时，它工作正常出现错误。

>>> a = foo.delay(20)     # works 
>>> a = foo.delay(1000)   # fails

如果您有任何想法并提前致谢，请提出如何解决这个问题。

Answer 1

我终于在来自redis user groups

的IRC和Greg的帮助下弄明白了

通过将Mac OSX Yosemite上已打开文件的限制更改为65536，问题得以解决。

我希望有一天能帮到某人。

与Celery组或链一起使用时Redis丢失连接 - 抛出ERROR / MainProcess]与Redis的连接丢失：立即重试（0/20）

1 个答案: