如何为使用Python在Direct Runner上运行的Apache Beam管道设置日志/打印

时间:2019-09-10 09:37:50

标签: python logging google-cloud-dataflow apache-beam

在使用直接流道的python,apap梁管道中进行编码时,我无法显示日志或打印内容。

import logging

class PrintFn(beam.DoFn):
    def process(self, element):
        print(element)
         logging.error(element)
        return [element]

def run(argv=None):
    lines = p | 'receive_data' >> beam.io.ReadFromPubSub(
            subscription=known_args.in_topic).with_input_types(str) \
            | 'decode' >> beam.Map(lambda x: x.decode('utf-8')) \
            | 'jsonload' >> beam.Map(lambda x: json.loads(x)) \
            |'print' >> beam.ParDo(PrintFn())

0 个答案:

没有答案