无法使用数据流将流复制到GCS

时间:2018-06-27 07:39:11

标签: python google-cloud-storage google-cloud-dataflow

当我尝试将pcollection复制到数据流中的GCS时,出现以下错误。

  

ERROR:root:访问时出错   WriteToText / Write / WriteImpl / FinalizeWrite追溯(最近一次调用)   最后):文件   “ /usr/lib64/python2.7/site-packages/apache_beam/runners/runner.py”,   第273行,位于get_pvalue中       value_with_refcount = self._cache [self.key(pvalue)] KeyError:(u'WriteToText / Write / WriteImpl / DoOnce / Decode Values,无)

我的代码:

import apache_beam as beam
from apache_beam.io import WriteToText
with beam.Pipeline(options=options) as p:    
    lines = p | 'Read PubSub' >> beam.io.ReadStringsFromPubSub(INPUT_TOPIC)| beam.WindowInto(window.FixedWindows(5))

    lines | beam.io.WriteToText(known_args.copytogs)

0 个答案:

没有答案