Question

我有一个JSON文档，想要创建一个Avro架构来序列化数据和反序列化。

我编写了一个下面定义的JSON文档的Avro架构，但是当我根据架构序列化JSON数据时，SchemaParser会抛出异常。虽然我阅读了很多关于Avro及其数据类型但却无法解决问题的内容。

对于这个问题，我已经指定了SchemaParser抛出的JSON文档，Avro架构和异常。

1）JSON文件

{
"category": "test",
"values": [
    {
        "subscriberid": 87392,
        "simserialnumber": 923,
        "MCC": 33,
        "MNC": [
            {
                "mn": {"key1":"kunal","key2":"gupta"},
                "mc": 44
            }
        ],
        "countryiso": "IN",
        "operatorname": "vodadone"
    }
]
}

2）Avro Schema

{
 "type": "record",
 "namespace": "testavro.schema",
 "name": "test",
 "fields": [
{
  "type": "string",
  "name": "data_version"
},
{
  "type": "string",
  "name": "ip_address"
},
{
  "type": "string",
  "name": "category"
},
{
  "type": {
    "items": {
      "fields": [
        {
          "type": "int",
          "name": "simserialnumber"
        },
        {
          "type": "string",
          "name": "countryiso"
        },
        {
          "type": "int",
          "name": "MCC"
        },
        {
          "type": "int",
          "name": "subscriberid"
        },
        {
          "type": {
            "items": {
              "fields": [
                {
                  "fields": [
                    {
                      "type": "string",
                      "name": "key2"
                    },
                    {
                      "type": "string",
                      "name": "key1"
                    }
                  ],
                  "type": "record",
                  "name": "mn"
                },
                {
                  "type": "int",
                  "name": "mc"
                }
              ],
              "type": "record",
              "name": "MNC_records"
            },
            "type": "array"
          },
          "name": "MNC"
        },
        {
          "type": "string",
          "name": "operatorname"
        }
      ],
      "type": "record",
      "name": "values_records"
    },
    "type": "array"
  },
  "name": "values"
}
]
}

3）SchemaParserException

SchemaParseException: Type property "{u'items': {u'fields': [{u'type': u'int', u'name': u'simserialnumber'}, {u'type': u'string', u'name': u'countryiso'}, {u'type': u'int', u'name': u'MCC'}, {u'type': u'int', u'name': u'subscriberid'}, {u'type': {u'items': {u'fields': [{u'fields': [{u'type': u'string', u'name': u'key2'}, {u'type': u'string', u'name': u'key1'}], u'type': u'record', u'name': u'mn'}, {u'type': u'int', u'name': u'mc'}], u'type': u'record', u'name': u'MNC_records'}, u'type': u'array'}, u'name': u'MNC'}, {u'type': u'string', u'name': u'operatorname'}], u'type': u'record', u'name': u'values_records'}, u'type': u'array'}" not a valid Avro schema: Items schema ({u'fields': [{u'type': u'int', u'name': u'simserialnumber'}, {u'type': u'string', u'name': u'countryiso'}, {u'type': u'int', u'name': u'MCC'}, {u'type': u'int', u'name': u'subscriberid'}, {u'type': {u'items': {u'fields': [{u'fields': [{u'type': u'string', u'name': u'key2'}, {u'type': u'string', u'name': u'key1'}], u'type': u'record', u'name': u'mn'}, {u'type': u'int', u'name': u'mc'}], u'type': u'record', u'name': u'MNC_records'}, u'type': u'array'}, u'name': u'MNC'}, {u'type': u'string', u'name': u'operatorname'}], u'type': u'record', u'name': u'values_records'}) not a valid Avro schema: Type property "{u'items': {u'fields': [{u'fields': [{u'type': u'string', u'name': u'key2'}, {u'type': u'string', u'name': u'key1'}], u'type': u'record', u'name': u'mn'}, {u'type': u'int', u'name': u'mc'}], u'type': u'record', u'name': u'MNC_records'}, u'type': u'array'}" not a valid Avro schema: Items schema ({u'fields': [{u'fields': [{u'type': u'string', u'name': u'key2'}, {u'type': u'string', u'name': u'key1'}], u'type': u'record', u'name': u'mn'}, {u'type': u'int', u'name': u'mc'}], u'type': u'record', u'name': u'MNC_records'}) not a valid Avro schema: Type property "record" not a valid Avro schema: Could not make an Avro Schema object from record. (known names: [u'testavro.schema.MNC_records', u'testavro.schema.test', u'testavro.schema.values_records']) (known names: [u'testavro.schema.MNC_records', u'testavro.schema.test', u'testavro.schema.values_records'])

请帮助我，克服这个问题对我很好。我花了整整一天时间使用这个JSON和Avro架构，但无法成功。

Answer 1

您的items中的MNC字段的values_record类型内部似乎有拼写错误。在新对象中包含mn的定义有效：

{
  "type": "record",
  "namespace": "testavro.schema",
  "name": "test",
  "fields": [
    {
      "type": "string",
      "name": "data_version"
    },
    {
      "type": "string",
      "name": "ip_address"
    },
    {
      "type": "string",
      "name": "category"
    },
    {
      "type": {
        "items": {
          "fields": [
            {
              "type": "int",
              "name": "simserialnumber"
            },
            {
              "type": "string",
              "name": "countryiso"
            },
            {
              "type": "int",
              "name": "MCC"
            },
            {
              "type": "int",
              "name": "subscriberid"
            },
            {
              "type": {
                "items": {
                  "fields": [
                    {
                      "type": {
                        "type": "record",
                        "fields": [
                          {
                            "type": "string",
                            "name": "key2"
                          },
                          {
                            "type": "string",
                            "name": "key1"
                          }
                        ],
                        "name": "Mn"
                      },
                      "name": "mn"
                    },
                    {
                      "type": "int",
                      "name": "mc"
                    }
                  ],
                  "type": "record",
                  "name": "MNC_records"
                },
                "type": "array"
              },
              "name": "MNC"
            },
            {
              "type": "string",
              "name": "operatorname"
            }
          ],
          "type": "record",
          "name": "values_records"
        },
        "type": "array"
      },
      "name": "values"
    }
  ]
}

从包含复杂JSON的映射（键值对）创建Avro架构

1 个答案: