Question

错误：

Exception in thread "main" java.lang.RuntimeException
{"errors":[{"debugInfo":"generic::failed_precondition: Repeated record added outside of an array.","reason":"invalid"}],"index":0}

语言： Scala

Gradle bigquery依赖：

compile "com.google.apis:google-api-services-bigquery:v2-rev326-1.22.0"

生成表架构的代码：

import scala.collection.JavaConversions._
val orderTableSchema = new TableSchema()
orderTableSchema.setFields(Seq(
  new TableFieldSchema().setName("revisionId").setType("STRING").setMode("REQUIRED"),
  new TableFieldSchema().setName("executions").setType("RECORD").setMode("REPEATED").setFields(Seq(
    new TableFieldSchema().setName("ts").setType("INTEGER"),
    new TableFieldSchema().setName("price").setType("FLOAT"),
    new TableFieldSchema().setName("qty").setType("INTEGER")
  ))
))

使用正确的执行模式列成功创建表，如BigQuery web ui：

所示

revisionId          STRING  REQUIRED    
executions          RECORD  REPEATED
executions          RECORD  REPEATED
executions.ts       INTEGER NULLABLE
executions.price    FLOAT   NULLABLE
executions.qty      INTEGER NULLABLE

插入失败的数据的代码：

import scala.collection.JavaConversions._
val row = new TableRow()
  .set("revisionId", "revision1")
  .set("executions", Seq(
    new TableRow().set("ts", 1L).set("price", 100.01).set("qty", 1000)
  ))
val content = new TableDataInsertAllRequest().setRows(Seq(
  new TableDataInsertAllRequest.Rows().setJson(row)
))
val insertAll = bigQuery.tabledata().insertAll(projectId, datasetId, "order", content)
val insertResponse = insertAll.execute()

if (insertResponse.getInsertErrors != null) {
  insertResponse.getInsertErrors.foreach(println)
  // this prints:
  // {"errors":[{"debugInfo":"generic::failed_precondition: Repeated record added outside of an array.","reason":"invalid"}],"index":0}

  // throw to just terminate early for demo
  throw new RuntimeException()
}

Answer 1

发现问题。

scala-to-java集合转换乱七八糟的东西。我不得不使用JavaConversions.seqAsJavaList显式添加转换，它神奇地开始工作

val row = new TableRow()
  .set("revisionId", s"revisionId$v")
  .set("executions", JavaConversions.seqAsJavaList(Seq(
    new TableRow().set("ts", 1L).set("price", 100.01).set("qty", 1000),
    new TableRow().set("ts", 2L).set("price", 100.02).set("qty", 2000),
    new TableRow().set("ts", 3L).set("price", 100.03).set("qty", 3000)
  )))

bigquery在数组外添加重复记录

1 个答案: