Jolt递归变换如何展平

时间:2015-12-07 17:08:06

标签: elasticsearch jolt

我有以下Elasticsearch结果:

{
    "took": 16,
    "timed_out": false,
    "_shards": {
        "total": 5,
        "successful": 5,
        "failed": 0
    },
    "hits": {
        "total": 77753,
        "max_score": 0,
        "hits": []
    },
    "aggregations": {
        "my_agg1": {
            "doc_count_error_upper_bound": 0,
            "sum_other_doc_count": 0,
            "buckets": [
                {
                    "key": "a",
                    "doc_count": 77739,
                    "my_agg2": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": [
                            {
                                "key": 777,
                                "doc_count": 77721,
                                "my_agg3": {
                                    "doc_count_error_upper_bound": 0,
                                    "sum_other_doc_count": 0,
                                    "buckets": [
                                        {
                                            "key": "1234",
                                            "doc_count": 48889
                                        },
                                        {
                                            "key": "5678",
                                            "doc_count": 25439
                                        },
                                        {
                                            "key": "9101",
                                            "doc_count": 3193
                                        },
                                        {
                                            "key": "9111",
                                            "doc_count": 196
                                        }
                                    ]
                                }
                            },
                            {
                                "key": 888,
                                "doc_count": 17,
                                "my_agg3": {
                                    "doc_count_error_upper_bound": 0,
                                    "sum_other_doc_count": 0,
                                    "buckets": [
                                        {
                                            "key": "1234",
                                            "doc_count": 9
                                        },
                                        {
                                            "key": "1233",
                                            "doc_count": 4
                                        },
                                        {
                                            "key": "1244",
                                            "doc_count": 3
                                        },
                                        {
                                            "key": "1230",
                                            "doc_count": 1
                                        }
                                    ]
                                }
                            },
                            {
                                "key": 999,
                                "doc_count": 1,
                                "my_agg3": {
                                    "doc_count_error_upper_bound": 0,
                                    "sum_other_doc_count": 0,
                                    "buckets": [
                                        {
                                            "key": "1261",
                                            "doc_count": 1
                                        }
                                    ]
                                }
                            }
                        ]
                    }
                },
                {
                    "key": "s",
                    "doc_count": 6,
                    "my_agg2": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": [
                            {
                                "key": 888,
                                "doc_count": 6,
                                "my_agg3": {
                                    "doc_count_error_upper_bound": 0,
                                    "sum_other_doc_count": 0,
                                    "buckets": [
                                        {
                                            "key": "1200",
                                            "doc_count": 4
                                        },
                                        {
                                            "key": "1210",
                                            "doc_count": 1
                                        },
                                        {
                                            "key": "1216",
                                            "doc_count": 1
                                        }
                                    ]
                                }
                            }
                        ]
                    }
                }
            ]
        }
    }
}

我想这样压扁一张桌子......

a, 777, 1234
a, 777, 5678
a, 777, 9101
a, 777, 9111
a, 888, 1234
a, 888, 1233
a, 888, 1244
a, 888, 1230
a, 999, 1261
s, 888, 1200
s, 888, 1210
s, 888, 1216

是否可以使用颠簸?

2 个答案:

答案 0 :(得分:2)

实际上,它可以使“csv”样式输出,但它需要两个Shifts。第一个转换按“agg”建立数据,然后第二个转换为输出数组。

规格

[
  {
    "operation": "shift",
    "spec": {
      "aggregations": {
        "my_agg1": {
          "buckets": {
            "*": {
              "my_agg2": {
                "buckets": {
                  "*": {
                    "my_agg3": {
                      "buckets": {
                        "*": {
                          "key": "agg3[]",
                          "@(3,key)": "agg2[]",
                          "@(6,key)": "agg1[]"
                        }
                      }
                    }
                  }
                }
              }
            }
          }
        }
      }
    }
  },
  {
    "operation": "shift",
    "spec": {
      "agg1": {
        "*": "[&][0]"
      },
      "agg2": {
        "*": "[&][1]"
      },
      "agg3": {
        "*": "[&][2]"
      }
    }
  }
]

制作

[ 
  [ "a", 777, "1234" ], 
  [ "a", 777, "5678" ], 
  [ "a", 777, "9101" ], 
  [ "a", 777, "9111" ], 
  [ "a", 888, "1234" ], 
  [ "a", 888, "1233" ], 
  [ "a", 888, "1244" ], 
  [ "a", 888, "1230" ], 
  [ "a", 999, "1261" ], 
  [ "s", 888, "1200" ], 
  [ "s", 888, "1210" ], 
  [ "s", 888, "1216" ] 
]

答案 1 :(得分:1)

的Ish。 “递归”不是问题,输出表/ csv格式是。它可以产生类似

的输出
{
  "a" : {
    "777" : [ "1234", "5678", "9101", "9111" ],
    "888" : [ "1234", "1233", "1244", "1230" ],
    "999" : [ "1261" ]
  },
  "s" : {
    "888" : [ "1200", "1210", "1216" ]
  }
}

鉴于此规范

[
  {
    "operation": "shift",
    "spec": {
      "aggregations": {
        "my_agg1": {
          "buckets": {
            "*": {
              "my_agg2": {
                "buckets": {
                  "*": {
                    "my_agg3": {
                      "buckets": {
                        "*": {
                          "key": "@(7,key).@(4,key)[]"
                        }
                      }
                    }
                  }
                }
              }
            }
          }
        }
      }
    }
  }
]