Question

我正在尝试配置Luigi的重试机制，以便重试失败的任务几次。但是，当任务成功重试时，Luigi退出失败了：

===== Luigi Execution Summary =====

Scheduled 3 tasks of which:
* 2 ran successfully:
    - 1 FailOnceThenSucceed(path=/tmp/job-id-18.subtask)
    - 1 MasterTask(path=/tmp/job-id-18)
* 1 failed:
    - 1 FailOnceThenSucceed(path=/tmp/job-id-18.subtask)

This progress looks :( because there were failed tasks

所以问题是：我如何配置Luigi（我已经安装了2.3.3版本的pip安装），这样当任务失败一次，但随后重试成功，那么Luigi将成功退出{{1}用This progress looks :)代替失败？

这是我提出的最小调度程序和工作程序配置，以及演示行为的任务：

This progress looks :(

mytasks.py：

[scheduler]
retry_count = 3
retry-delay = 1

[worker]
keep_alive=true

执行示例：

import luigi class FailOnceThenSucceed(luigi.Task): path = luigi.Parameter() def output(self): return luigi.LocalTarget(self.path) def run(self): failmarker = luigi.LocalTarget(self.path + ".hasfailedonce") if failmarker.exists(): with self.output().open('w') as target: target.write('OK') else: with failmarker.open('w') as marker: marker.write('Failed') raise RuntimeError("Failed once") class MasterTask(luigi.Task): path = luigi.Parameter() def requires(self): return FailOnceThenSucceed(path=self.path + '.subtask') def output(self): return luigi.LocalTarget(self.path) def run(self): with self.output().open('w') as target: target.write('OK')

Answer 1

这是Luigi的一个老问题 - 成功重试的任务在失败时没有被标记，然后成功重试： https://github.com/spotify/luigi/issues/1932

在2.7.2版本中已修复： https://github.com/spotify/luigi/releases/tag/2.7.2

我建议您升级到最新的Luigi版本，即运行pip install -U luigi。

如何正确配置Luigi任务重试？

1 个答案: