Question

我对Apache Beam Pipeline相对较新，我试图修改现有的梁管道代码，方法是在'Data'类中插入一个参数并检索它以在后续步骤中进行进一步的计算，如下所示。

public class Data {  
  @JsonProperty("loss")
  private double loss;

  public double getLoss() {
    return loss;
  }

  public void setLoss(double loss) {
    this.loss = loss;
  }
}

我还对data.json中的架构进行了更改，如下所示：

final TableSchema dataSchema = SchemaLoader.load("/data.json");

writeIntoBigQuery(options.getDataTableName(), dataSchema, Data, new InputToTableRow());

然而，我仍然遇到运行时错误（我只复制了前几行，其余的似乎都在抱怨或多或少相同的事情）。最初我认为这可能是因为我的BigQuery表没有'丢失'字段作为表模式的一部分并去添加它但它仍然发生..代码中的其他地方我应该添加或修改以使这个错误消失了吗？或者我应该做些什么来获得有关如何排除故障的更多细节？

java.lang.RuntimeException: java.io.IOException: Insert failed: [{"errors":[{"debugInfo":"generic::not_found: no such field.","location":"loss","message":"no such field.","reason":"invalid"}],"index":0}]
        org.apache.beam.sdk.io.gcp.bigquery.StreamingWriteFn.flushRows(StreamingWriteFn.java:106)
        org.apache.beam.sdk.io.gcp.bigquery.StreamingWriteFn.finishBundle(StreamingWriteFn.java:82)
Caused by: java.io.IOException: Insert failed: [{"errors":[{"debugInfo":"generic::not_found: no such field.","location":"loss","message":"no such field.","reason":"invalid"}],"index":0}]

java.io.IOException遇到插入错误

0 个答案: