Kafka JDBC接收器不处理空值

时间:2019-06-05 08:48:11

标签: apache-kafka avro apache-kafka-connect

我正在尝试使用Kafka JDBC Sink连接器插入数据,但它会返回此异常。

org.apache.kafka.connect.errors.DataException: Invalid null value for required INT64 field

记录具有以下架构:

[
  {
    "schema": {
      "type": "struct",
      "fields": [
        {
          "type": "int64",
          "field": "ID"
        },
        {
          "type": "int64",
          "field": "TENANT_ID"
        },
        {
          "type": "string",
          "field": "ITEM"
        },
        {
          "type": "int64",
          "field": "TIPO"
        },
        {
          "type": "int64",
          "field": "BUSINESS_CONCEPT"
        },
        {
          "type": "string",
          "field": "ETIQUETA"
        },
        {
          "type": "string",
          "field": "VALOR"
        },
        {
          "type": "string",
          "field": "GG_T_TYPE"
        },
        {
          "type": "string",
          "field": "GG_T_TIMESTAMP"
        },
        {
          "type": "string",
          "field": "TD_T_TIMESTAMP"
        },
        {
          "type": "string",
          "field": "POS"
        }
      ]
    },
    "payload": {
      "ID": 298457,
      "TENANT_ID": 83,
      "ITEM": "0-0-0",
      "TIPO": 4,
      "BUSINESS_CONCEPT": null,
      "ETIQUETA": "¿Cuándo ha ocurrido?",
      "VALOR": "2019-05-31T10:33:00Z",
      "GG_T_TYPE": "I",
      "GG_T_TIMESTAMP": "2019-05-31 14:35:19.002221",
      "TD_T_TIMESTAMP": "2019-06-05T10:46:55.0106",
      "POS": "00000000530096832544"
    }
  }
]

如您所见,值BUSINESS_CONCEPT可以是null。它是唯一的null值,所以我想认为异常是由于该字段引起的。如何使接收器将值插入为null

1 个答案:

答案 0 :(得分:2)

您需要更改

的定义
{
  "type": "int64",
  "field": "BUSINESS_CONCEPT"
}

{
  "type": ["null", "int64"],
  "field": "BUSINESS_CONCEPT"
}

,以便将BUSINESS_CONCEPT视为可选字段。