Question

我最近将现有的管道从数据流1.x升级到数据流2.x，我看到一个对我没有意义的错误。我将下面的相关代码，然后包括我看到的错误。

// This is essentially the final step in our pipeline, where we write
// one of the side outputs from the pipeline to a BigQuery table

results.get(matchedTag)
  .apply("CountBackfill", Count.<String>perElement())
  .apply("ToReportRow", ParDo.of(new ToReportRow()))
  // at this point, there is now a PCollection<TableRow>
  .apply("WriteReport", BigQueryIO.writeTableRows()
    .to(reportingDataset + ".AttributeBackfill_" + dayStr)
    .withSchema(ReportSchema.get())
    .withWriteDisposition(BigQueryIO.Write.WriteDisposition.WRITE_TRUNCATE)
    .withCreateDisposition(BigQueryIO.Write.CreateDisposition.CREATE_IF_NEEDED));


/*
* Create a TableRow from a key/value pair
*/
public static class ToReportRow extends DoFn<KV<String, Long>, TableRow> {

  private static final long serialVersionUID = 1L;

  @ProcessElement
  public void processElement(ProcessContext c) throws InterruptedException {
    KV<String, Long> row = c.element();

    c.output(new TableRow()
      .set(ReportSchema.ID, row.getKey())
      .set(ReportSchema.COUNT, row.getValue()));
  }
}

这是我看到的错误：

线程“main”中的异常java.lang.NoSuchMethodError： com.google.common.base.Preconditions.checkArgument（ZLjava /郎/字符串; Ljava /郎/对象; Ljava /郎/对象;）V 在 org.apache.beam.sdk.io.gcp.bigquery.BigQueryIO $ Write.expand（BigQueryIO.java:1426）在 org.apache.beam.sdk.io.gcp.bigquery.BigQueryIO $ Write.expand（BigQueryIO.java:989）在org.apache.beam.sdk.Pipeline.applyInternal（Pipeline.java:525）at org.apache.beam.sdk.Pipeline.applyTransform（Pipeline.java:479）at org.apache.beam.sdk.values.PCollection.apply（PCollection.java:297）at com.prod.merge.DailyUniqueProfiles.buildPipeline（DUP.java:106）在com.prod.merge.MergePipeline.main（MergePipeline.java:91）

.apply("WriteReport", BigQueryIO.writeTableRows()行是DUP.java中的第106行，所以这就是我怀疑在某种程度上错误的行。

关于问题可能是什么的任何想法？

Answer 1

此问题的解决方案最终成为maven依赖项。添加以下依赖项并使用mvn重新编译后，错误就消失了。

<dependency>
  <groupId>com.google.guava</groupId>
  <artifactId>guava</artifactId>
  <version>22.0</version>
</dependency>

Answer 2

看起来旧版本的番石榴正被传递依赖所吸引。以下对依赖项的更改对我有用。

<a class="up-not-clicked" id="{{ puzzle.ID }}">&uArr;</a>

Apache Beam，BigQueryIO.WriteTableRows（）上的NoSuchMethodError？

2 个答案: