Question

我的数据流作业失败，并出现以下错误：

INFO:root:2018-10-15T18:55:37.417Z: JOB_MESSAGE_ERROR: Workflow failed. 
Causes: S17:fold2/Write/WriteImpl/WindowInto(WindowIntoFn)+write instances fold2/Write/WriteImpl/GroupByKey/Reify+write instances fold2/Write/WriteImpl/GroupByKey/Write failed., 
A work item was attempted 4 times without success. 
Each time the worker eventually lost contact with the service. The work item was attempted on: 
  yuri-nine-gag-recommender-10151140-3kmq-harness-mdgd,
  yuri-nine-gag-recommender-10151140-3kmq-harness-mdgd,
  yuri-nine-gag-recommender-10151140-3kmq-harness-41dd,
  yuri-nine-gag-recommender-10151140-3kmq-harness-mdgd

浏览日志仅显示一个错误：

An exception was raised when trying to execute the workitem 6479210647275353150 : 
Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/batchworker.py", line 642, in do_work work_executor.execute() 
File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/executor.py", line 158, in execute op.finish() 
File "dataflow_worker/shuffle_operations.py", line 144, in dataflow_worker.shuffle_operations.ShuffleWriteOperation.finish def finish(self): 
File "dataflow_worker/shuffle_operations.py", line 145, in dataflow_worker.shuffle_operations.ShuffleWriteOperation.finish with self.scoped_finish_state: 
File "dataflow_worker/shuffle_operations.py", line 147, in dataflow_worker.shuffle_operations.ShuffleWriteOperation.finish self.writer.__exit__(None, None, None) 
File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/shuffle.py", line 599, in __exit__ self.writer.Close() 
File "third_party/windmill/shuffle/python/shuffle_client.pyx", line 202, in shuffle_client.PyShuffleWriter.Close IOError: Shuffle close failed: FAILED_PRECONDITION: Precondition check failed.

有什么想法吗？

Answer 1

我终于解决了这个问题，方法是删除各种代码段，打印大量日志并再次运行作业。原来，我有一个正则表达式，对于一个特定的条目来说很夸张。不幸的是，数据流日志根本没有帮助。

数据流作业失败，原因：随机关闭失败：FAILED_PRECONDITION：前提条件检查失败

1 个答案: