侧面输入用于收集

时间:2019-08-28 08:02:56

标签: apache-beam

我想在管道中动态输入字符串值。 (JsonSchema值)。 它来自另一个管道。 PCollection<String>

 row.apply("Inserting to BQ table", BigQueryIO.writeTableRows()
                                  //facing issues here
                                  .withJsonSchema("")
                                  .withCreateDisposition(
                                                BigQueryIO.Write.CreateDisposition.CREATE_NEVER)
                                  .withWriteDisposition(
                                                BigQueryIO.Write.WriteDisposition.WRITE_APPEND)
                                  // .withExtendedErrorInfo()
                                  // .withMethod(BigQueryIO.Write.Method.STREAMING_INSERTS)
                                  .withMethod(BigQueryIO.Write.Method.FILE_LOADS)
                                  .withTriggeringFrequency(Duration.standardMinutes(1))
                                  // .withFailedInsertRetryPolicy(InsertRetryPolicy.retryTransientErrors())
                                  .withOptimizedWrites().withNumFileShards(5)
                                  .to(options.getBqTargetTableName()));

0 个答案:

没有答案