与Java中的BigQuery同步“在数组外部添加重复的记录”

Question

我正在尝试使用Java向BigQuery插入一行。 我要插入的实体的字段是双重嵌套的。

生成适合BigQuery的实体

ObjectMapper mapper = new ObjectMapper();
String barcodeDetailsJSON = order.getBarcodeDetailsJSON();
List<StateForBQ> stateForBQList = new ArrayList<StateForBQ>();
for (State state : order.getStates()) {
  StateForBQ stateForBQ = new StateForBQ(state);
  stateForBQ.setSetOn(new Date(stateForBQ.getSetOn().getTime()/1000));
  stateForBQList.add(stateForBQ);
}
  List<BarcodeDetailForBQ> barcodeDetailForBQList = getBarcodeDetailsFromBarcodeDetailsJSON(barcodeDetailsJSON, order.getIsGrouped());

不执行以下操作，状态将设置为null。 （状态是嵌套实体）

  List<Map<String, Object>> stateMap = 
      mapper.convertValue(stateForBQList, new TypeReference<List<Map<String, Object>>>() {});

不执行以下操作，barcodeDetails设置为null。 （BarcodeDetails是双重嵌套实体）

  List<Map<String, Object>> barcodeMapList = 
      mapper.convertValue(barcodeDetailForBQList, new TypeReference<List<Map<String, Object>>>() {});

在没有关注的情况下，productPriceDetails，productDetails，cgst，sgst被设置为null

  for (Map<String, Object> barcodeMap : barcodeMapList) {
      barcodeMap.put("productPriceDetails", mapper.convertValue(barcodeMap.get("productPriceDetails"), new TypeReference<Map<String, Object>>() {}));
      barcodeMap.put("productDetails", mapper.convertValue(barcodeMap.get("productDetails"), new TypeReference<Map<String, Object>>() {}));
      barcodeMap.put("cgst", mapper.convertValue(barcodeMap.get("cgst"), new TypeReference<Map<String, Object>>() {}));
      barcodeMap.put("sgst", mapper.convertValue(barcodeMap.get("sgst"), new TypeReference<Map<String, Object>>() {}));
}

准备行内容

  Map<String, Object> rowContent = new HashMap<>();
  rowContent.put("orderId", order.getOrderId());
  rowContent.put("customerId", order.getCustomerId());
  rowContent.put("barcodeDetails", barcodeMapList);
  rowContent.put("states", stateMap);

插入BigQuery

Gson gson = new Gson();
BigQuery bigquery = BigQueryOptions.getDefaultInstance().getService();
String datasetName = "Latest_Data";
String tableName= "ORDER_TEMP";
  // [START bigquery_table_insert_rows]
  TableId tableId = TableId.of(datasetName, tableName);
  // Values of the row to insert

String barcodeDetailsJSON = order.getBarcodeDetailsJSON();
  List<BarcodeDetailForBQ> barcodeDetailForBQList = new OrderForBQ().getBarcodeDetailsFromBarcodeDetailsJSON(barcodeDetailsJSON, order.getIsGrouped());

  String recordsContentString = gson.toJson(rowContent);

  InsertAllResponse response =
      bigquery.insertAll(
          InsertAllRequest.newBuilder(tableId)
              .addRow(""+orderId, rowContent)
              // More rows can be added in the same RPC by invoking .addRow() on the builder
              .build());

  if (response.hasErrors()) {
    // If any of the insertions failed, this lets you inspect the errors
    for (Entry<Long, List<BigQueryError>> entry : response.getInsertErrors().entrySet()) {
      // inspect row error
    }
  }

以下是我得到的答复。

{
  insertErrors: {
    0: [
      {
        reason: "invalid",
        location: "",
        message: "Repeated record added outside of an array."
      }
    ]
  }
}