我正在尝试使用Protocol Buffer在本地集群上运行一个简单的管道来在Beam函数之间传递数据。 com.google.protobuf:protobuf-java包含在FatJar中。
如果我运行它，一切正常：
java -jar target/dataflow-test-1.0-SNAPSHOT.jar \
--runner=org.apache.beam.runners.flink.FlinkRunner \
--input=/tmp/kinglear.txt --output=/tmp/wordcounts.txt
但尝试在flink群集上运行时失败了：
flink run target/dataflow-test-1.0-SNAPSHOT.jar \
--runner=org.apache.beam.runners.flink.FlinkRunner \
--input=/tmp/kinglear.txt --output=/tmp/wordcounts.txt
准备运行项目： https://github.com/orian/beam-flink-local-cluster
Beam版本是0.3-Snapshot（前沿），它使用Flink版本1.0.3，我的本地群集运行版本1.0.3。 Flink使用protobuf-java 2.5.0。
程序结束时出现以下异常：
java.lang.NoSuchMethodError: com.google.protobuf.ExtensionRegistry.getAllImmutableExtensionsByExtendedType(Ljava/lang/String;)Ljava/util/Set;
at org.apache.beam.sdk.coders.protobuf.ProtobufUtil.recursivelyAddDescriptors(ProtobufUtil.java:129)
at org.apache.beam.sdk.coders.protobuf.ProtobufUtil.getRecursiveDescriptorsForClass(ProtobufUtil.java:63)
at org.apache.beam.sdk.coders.protobuf.ProtobufUtil.verifyDeterministic(ProtobufUtil.java:92)
at org.apache.beam.sdk.coders.protobuf.ProtoCoder.verifyDeterministic(ProtoCoder.java:258)
at org.apache.beam.sdk.transforms.GroupByKey.validate(GroupByKey.java:196)
at org.apache.beam.sdk.transforms.GroupByKey.validate(GroupByKey.java:121)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:399)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:307)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:154)
at org.apache.beam.sdk.transforms.Combine$PerKey.apply(Combine.java:1851)
at org.apache.beam.sdk.transforms.Combine$PerKey.apply(Combine.java:1740)
at org.apache.beam.sdk.runners.PipelineRunner.apply(PipelineRunner.java:76)
at org.apache.beam.runners.flink.FlinkRunner.apply(FlinkRunner.java:196)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:400)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:307)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:154)
at org.apache.beam.sdk.transforms.Count$PerElement.apply(Count.java:115)
at org.apache.beam.sdk.transforms.Count$PerElement.apply(Count.java:100)
at org.apache.beam.sdk.runners.PipelineRunner.apply(PipelineRunner.java:76)
at org.apache.beam.runners.flink.FlinkRunner.apply(FlinkRunner.java:196)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:400)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:307)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:154)
at com.mycompany.dataflow.WordCount$CountWords.apply(WordCount.java:74)
at com.mycompany.dataflow.WordCount$CountWords.apply(WordCount.java:63)
at org.apache.beam.sdk.runners.PipelineRunner.apply(PipelineRunner.java:76)
at org.apache.beam.runners.flink.FlinkRunner.apply(FlinkRunner.java:196)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:400)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:307)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:154)
at com.mycompany.dataflow.WordCount.main(WordCount.java:118)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:505)
at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:403)
at org.apache.flink.client.program.Client.runBlocking(Client.java:248)
at org.apache.flink.client.CliFrontend.executeProgramBlocking(CliFrontend.java:866)
at org.apache.flink.client.CliFrontend.run(CliFrontend.java:333)
at org.apache.flink.client.CliFrontend.parseParameters(CliFrontend.java:1192)
at org.apache.flink.client.CliFrontend.main(CliFrontend.java:1243)