由于数据流运行程序未关闭文件系统编写器,因此管道继续运行。 NotImplementedError

时间:2019-08-15 08:48:53

标签: google-cloud-dataflow

我正在Google Cloud Platform上使用python运行数据流作业(Apache Beam 2.12.0)。管道没有终止,并且继续运行。

问题与 https://issues.apache.org/jira/browse/BEAM-7266 它没有解决,并说“再次见面时打开”。它还表示文件编写器没有关闭。

只有一个错误日志:

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/batchworker.py", line 649, in do_work
    work_executor.execute()
  File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/executor.py", line 178, in execute
    op.finish()
  File "dataflow_worker/native_operations.py", line 93, in dataflow_worker.native_operations.NativeWriteOperation.finish
    def finish(self):
  File "dataflow_worker/native_operations.py", line 94, in dataflow_worker.native_operations.NativeWriteOperation.finish
    with self.scoped_finish_state:
  File "dataflow_worker/native_operations.py", line 95, in dataflow_worker.native_operations.NativeWriteOperation.finish
    self.writer.__exit__(None, None, None)
  File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/nativefileio.py", line 465, in __exit__
    self.file.close()
  File "/usr/local/lib/python2.7/dist-packages/apache_beam/io/filesystemio.py", line 202, in close
    self._uploader.finish()
  File "/usr/local/lib/python2.7/dist-packages/apache_beam/io/gcp/gcsio.py", line 606, in finish
    raise self._upload_thread.last_error  # pylint: disable=raising-bad-type
NotImplementedError: offset: 0, whence: 0, position: 51518, last: 0

0 个答案:

没有答案