Question

我的芹菜任务如下：

@celery.task(name='tasks.ht_run', default_retry_delay=30, max_retries=15, time_limit=1)  
def ht_run(str_command):
    try:
        f = os.popen(str_command)
        output = f.read()
        if output == '':
            raise Exception
    except Exception as exc:
        raise ht_run.retry(exc=exc)

    return output.split('\n')

它被称为： appserver.ht_run.delay（字符串）

虽然我希望它在超时时重试，但是如果失败了。在Celery窗口中，我收到以下错误：

[2014-12-10 11:50:22,128: ERROR/MainProcess] Task tasks.ht_run[6a83793a-8bd6-47fc-bf74-0b673bf961f2] raised unexpected: TimeLimitExceeded(1,)
Traceback (most recent call last):
  File "/Users/andy.terhune/cgenv/lib/python2.6/site-packages/billiard/pool.py", line 639, in on_hard_timeout
    raise TimeLimitExceeded(job._timeout)
TimeLimitExceeded: TimeLimitExceeded(1,)
[2014-12-10 11:50:22,128: ERROR/MainProcess] Hard time limit (1s) exceeded for tasks.ht_run[6a83793a-8bd6-47fc-bf74-0b673bf961f2]
[2014-12-10 11:50:23,644: ERROR/MainProcess] Process 'Worker-28' pid:2081 exited with 'signal 9 (SIGKILL)'

如何让它超时并在重试时重试？

Answer 1

感谢user2097159，

尝试了一个软超时，就像一个魅力，

@celery.task(
    name='tasks.ht_run', 
    default_retry_delay=30,
    max_retries=15,
    soft_time_limit=1)
def ht_run(str_command):
    try:
        f = os.popen(str_command)
        output = f.read()
        if output == '':
            raise Exception
    except Exception as exc:
        raise ht_run.retry(exc=exc)

    return output.split('\n')

...

[2014-12-10 12:16:55,580: WARNING/MainProcess] Soft time limit (1s) exceeded for tasks.ht_run[f31bbd15-e755-440c-9261-cd0864ceb3a9]
[2014-12-10 12:16:55,603: INFO/MainProcess] Received task: tasks.ht_run[f31bbd15-e755-440c-9261-cd0864ceb3a9] eta:[2014-12-10 18:17:25.593613+00:00]
[2014-12-10 12:16:55,603: DEBUG/MainProcess] basic.qos: prefetch_count->121
[2014-12-10 12:16:56,897: INFO/MainProcess] Task tasks.ht_run[f31bbd15-e755-440c-9261-cd0864ceb3a9] retry: Retry in 30s: SoftTimeLimitExceeded()

Answer 2

现在在4.0版中，您可以为装饰器中的特定异常指定autoretry。以下是文档中的示例：

from twitter.exceptions import FailWhaleError

@app.task(autoretry_for=(FailWhaleError,))
def refresh_timeline(user):
    return twitter.refresh_timeline(user)

其中一个例外可以SoftTimeLimitExceeded来满足您的情况。

如何在硬超时重试芹菜任务？

2 个答案: