Question

我创建了一个测试用例来重现我在更复杂的工作流程中遇到的问题。我希望每次和弦标题发生错误时都会执行和弦回调的link_error。在提供的示例中，和弦标题是一个组，其由一个链组成。但是，在我的实现组中可以包含多个链和/或任务。

我的芹菜版 3.1.23 。

test.py：

from celery import Celery, chain, group, chord

app = Celery('test', backend='redis', broker='redis://localhost')

@app.task
def task1():
    print("Task 1 executed")

@app.task
def error_task():
    raise Exception("Mocked exception")

@app.task
def callback_task():
    print("Callback executed")


@app.task
def link_error_task(*args, **kwargs):
    print("Link error executed")


@app.task
def link_task():
    print("Link executed")


def main():
    c1 = chain(error_task.si(), task1.si())
    head = group([c1])
    callback = callback_task.si().set(link=[link_task], link_error=[link_error_task.s()])

    chord(head, callback).delay()

我希望发生以下事情：

1）芹菜工人开始在 c1 链中执行任务。

2）在 error_task 中达到异常后，芹菜停止执行链 c1 。

3）和弦标题的错误会传播到和弦回调。因此，回调执行将被标记为失败，最后将执行回调的 link_error 。

要执行此代码，请运行以下命令：

1号航站楼：

$ celery -A test worker --app=test:app --loglevel=info

2号航站楼：

$ python
Python 2.7.12 (default, Jun 29 2016, 14:05:02)
[GCC 4.2.1 Compatible Apple LLVM 7.3.0 (clang-703.0.31)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import test
>>> test.main()

芹菜产量将如下：

[2016-07-14 21:39:36,186: INFO/MainProcess] Received task: test.error_task[b9e7d006-43f9-4e9d-8391-ada74fb3bc4e]
[2016-07-14 21:39:36,193: ERROR/MainProcess] Task test.error_task[b9e7d006-43f9-4e9d-8391-ada74fb3bc4e] raised unexpected: Exception('Mocked exception',)
Traceback (most recent call last):
  File "/private/tmp/venv/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/private/tmp/venv/lib/python2.7/site-packages/celery/app/trace.py", line 438, in __protected_call__
    return self.run(*args, **kwargs)
  File "/private/tmp/test.py", line 11, in error_task
    raise Exception("Mocked exception")
Exception: Mocked exception

我们可以看到 link_error 未执行。但是，如果我们删除 c1 中的上一个任务，程序将按预期运行：

    def main():
        c1 = chain(error_task.si()) # last task removed
        head = group([c1])
        callback = callback_task.si().set(link=[link_task], link_error=[link_error_task.s()])

        chord(head, callback).delay()

输出将如下：

[2016-07-14 21:42:47,177: INFO/MainProcess] Received task: test.error_task[96d5cb6c-b059-452a-b8cf-73e54f495976]
[2016-07-14 21:42:47,185: ERROR/Worker-4] Chord 'b3f57200-a52f-426f-9814-202576ae3538' raised: "Dependency 96d5cb6c-b059-452a-b8cf-73e54f495976 raised Exception('Mocked exception',)"
Traceback (most recent call last):
  File "/private/tmp/venv/lib/python2.7/site-packages/celery/backends/base.py", line 580, in on_chord_part_return
    ret = j(timeout=3.0, propagate=propagate)
  File "/private/tmp/venv/lib/python2.7/site-packages/celery/result.py", line 691, in join_native
    raise value
Exception: Mocked exception
[2016-07-14 21:42:47,207: INFO/MainProcess] Received task: test.link_error_task[c3c4408b-17fc-4efa-b502-c7cdf3862240]
[2016-07-14 21:42:47,209: WARNING/Worker-3] Link error executed
[2016-07-14 21:42:47,210: ERROR/MainProcess] Task test.error_task[96d5cb6c-b059-452a-b8cf-73e54f495976] raised unexpected: Exception('Mocked exception',)
Traceback (most recent call last):
  File "/private/tmp/venv/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/private/tmp/venv/lib/python2.7/site-packages/celery/app/trace.py", line 438, in __protected_call__
    return self.run(*args, **kwargs)
  File "/private/tmp/test.py", line 11, in error_task
    raise Exception("Mocked exception")
Exception: Mocked exception
[2016-07-14 21:42:47,214: INFO/MainProcess] Task test.link_error_task[c3c4408b-17fc-4efa-b502-c7cdf3862240] succeeded in 0.00591249897843s: None

总之，问题似乎与链 c1 有关。如果任务后至少有一个任务触发错误，则回调的 link_error 将不会执行。

有人可以帮我解决这个问题吗？

当任务在和弦标题中失败时，不会执行link_error

0 个答案: