Question

我正在尝试使用 SplitRecord 处理器拆分包含嵌套记录的JSON文件。

这使我可以验证所提供的avro模式，并准备下一个阶段：使用ForkRecord展平每个记录

SplitRecord的输出总是给我一个空值，而不是预期的记录数组：

{"userid":"xxx","bookmarks":null}

下面是示例JSON

{
  "_id": {
    "oid": "5715fd7dc6b491db15a4243b"
  },
  "bookmarks": [
    {
      "_id": {
        "oid": "577d676eca6baf40f15d9c91"
      },
      "id": "10000XXXXXXW0007760",
      "creator": "player",
      "position": 42.96,
      "creationdate": {
        "date": "2016-05-27T19:19:02.379Z"
      }
    },
    {
      "_id": {
        "oid": "577d676eca6baf40f15d9c94"
      },
      "id": "ALIGXXXXXXXW0007944",
      "creator": "player",
      "position": 95.06,
      "creationdate": {
        "date": "2016-03-31T15:07:29.654Z"
      }
    }
  ],
  "lastaction": {
    "date": "2019-07-24T12:13:27.388Z"
  },
  "settings": {
    "preferredlanguage": "vost",
    "preferredvideoquality": "hd"
  },
  "userid": "xxxx",
  "maskedcontents": []
}

这是我的avro模式，存储在 AvroSchemaRegistry 中：

{
    "namespace": "nifi",
    "name": "bookmark",
    "type": "record",
    "fields": [
        { "name": "userid", "type": "string" },
        { "name": "bookmarks", "type": {
            "type": "record",
            "name": "bookmarks",
                "fields": [
                    { "name": "id", "type": "string" },
                    { "name": "creator", "type": "string" },
                    { "name": "position", "type": "float" }
                ]
            }
        }
    ]
}

SplitRecord处理器使用JsonRecordSetWriter，其角度配置为：

任何帮助将不胜感激！

Answer 1

我必须实现一个特定的常规处理器来克服nifi的局限性，这花了我很多时间。 Avro方案的管理仅限于最简单的情况，不适用于高级治疗。

Nifi：嵌套的JSON记录架构验证

1 个答案: