管道失败,并带有无效的名称空间字符串:“ //”

时间:2018-08-24 16:13:34

标签: java namespaces runtime-error google-cloud-dataflow apache-beam

上下文:使用Windows和GroupByKey阶段的数据流2.6.0的流管道

发生了什么:短时间后,管道不断发出以下错误:

java.lang.RuntimeException: Invalid namespace string: '//'

看堆栈跟踪,发生这种情况是因为StateNamespaceshttps://github.com/apache/beam/blob/master/runners/core-java/src/main/java/org/apache/beam/runners/core/StateNamespaces.java)试图向CoderUtils.decodeFromBase64发送一个空的命名空间字符串,从而创建了一个未捕获的EOF error,它本身启动CoderException

但是,这仅在云中发生。确实,查看堆栈跟踪记录可以发现,数据流专有代码在这里很危急。 (特别是直接跑步者从不使用StateNamespaces

这是堆栈跟踪:(完整堆栈跟踪在此处https://pastebin.com/xwh82pYx

 java.lang.RuntimeException: Invalid namespace string: '//'
        org.apache.beam.runners.core.StateNamespaces.fromString(StateNamespaces.java:270)
        com.google.cloud.dataflow.worker.WindmillTimerInternals.windmillTimerToTimerData(WindmillTimerInternals.java:264)
        com.google.cloud.dataflow.worker.StreamingModeExecutionContext$StepContext.lambda$getNextFiredTimer$1(StreamingModeExecutionContext.java:535)
        com.google.cloud.dataflow.worker.repackaged.com.google.common.collect.Iterators$7.transform(Iterators.java:750)
        com.google.cloud.dataflow.worker.repackaged.com.google.common.collect.TransformedIterator.next(TransformedIterator.java:47)
        com.google.cloud.dataflow.worker.StreamingModeExecutionContext$StepContext.getNextFiredTimer(StreamingModeExecutionContext.java:543)
        com.google.cloud.dataflow.worker.SimpleParDoFn.processTimers(SimpleParDoFn.java:445)
        com.google.cloud.dataflow.worker.SimpleParDoFn.processTimers(SimpleParDoFn.java:343)
        com.google.cloud.dataflow.worker.util.common.worker.ParDoOperation.finish(ParDoOperation.java:51)
        com.google.cloud.dataflow.worker.util.common.worker.MapTaskExecutor.execute(MapTaskExecutor.java:83)
        com.google.cloud.dataflow.worker.StreamingDataflowWorker.process(StreamingDataflowWorker.java:1227)
        com.google.cloud.dataflow.worker.StreamingDataflowWorker.access$1000(StreamingDataflowWorker.java:136)
        com.google.cloud.dataflow.worker.StreamingDataflowWorker$6.run(StreamingDataflowWorker.java:966)
        java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.beam.sdk.coders.CoderException: java.io.EOFException
        org.apache.beam.sdk.coders.InstantCoder.decode(InstantCoder.java:70)
        org.apache.beam.sdk.coders.InstantCoder.decode(InstantCoder.java:34)
        ...

如何解决此错误?


一些调查:

  • 似乎没有关于类似错误的文档
  • 这可能与flink运行器中的this issue有关,但是应该固定在梁2.5.0中

0 个答案:

没有答案