通过jolt变换复制具有不同配置的文档

时间:2017-12-15 17:14:15

标签: json jolt

我的目标是获取一个在子树中包含数组的输入文档,并将整个文档复制到文档的副本数组中,并在每个后续副本中设置该数组的各个值。

举个例子:

启动文件:

{
  "config": {
    "activeConfig": {
      "sourceDatabase": "test",
      "targetSites": [
        {
          "siteName": "location1",
          "targetDatabase": "devl",
          "siteShortName": "123"
        },
        {
          "siteName": "location2",
          "targetDatabase": "123",
          "siteShortName": "123"
        }
      ]
    }
  },
  "secondData": {
    "queries": [
      {
        "Tablename": "abc",
        "Query": "123"
      }
    ]
  }
}

预期产出:

[ {
  "config" : {
    "activeConfig" : {
      "sourceDatabase" : "test",
      "targetSites" : [ {
        "siteName" : "location1",
        "targetDatabase" : "devl",
        "siteShortName" : "123"
      }, {
        "siteName" : "location2",
        "targetDatabase" : "123",
        "siteShortName" : "123"
      } ],
      "currentSite" : {
        "siteName" : "location1",
        "targetDatabase" : "devl",
        "siteShortName" : "123"
      }
    }
  },
  "secondData" : {
    "queries" : [ {
      "Tablename" : "abc",
      "Query" : "123"
    } ]
  }
},
 {
  "config" : {
    "activeConfig" : {
      "sourceDatabase" : "test",
      "targetSites" : [ {
        "siteName" : "location1",
        "targetDatabase" : "devl",
        "siteShortName" : "123"
      }, {
        "siteName" : "location2",
        "targetDatabase" : "123",
        "siteShortName" : "123"
      } ],
      "currentSite" : {
        "siteName" : "location2",
        "targetDatabase" : "123",
        "siteShortName" : "123"
      }
    }
  },
  "secondData" : {
    "queries" : [ {
      "Tablename" : "abc",
      "Query" : "123"
    } ]
  }
} ]

我到目前为止的JOLT规范如下:

[
  {
    "operation": "shift",
    "spec": {
      "config": {
        "activeConfig": {
          "targetSites": {
            "*": {
              "@4": "[]",
              "@": "[].config.activeConfig.currentSite"
            }
          }
        }
      }
    }
  }
]

这让我很接近,但并不完全在那里。

[ {
  "config" : {
    "activeConfig" : {
      "sourceDatabase" : "test",
      "targetSites" : [ {
        "siteName" : "location1",
        "targetDatabase" : "devl",
        "siteShortName" : "123"
      }, {
        "siteName" : "location2",
        "targetDatabase" : "123",
        "siteShortName" : "123"
      } ]
    }
  },
  "secondData" : {
    "queries" : [ {
      "Tablename" : "abc",
      "Query" : "123"
    } ]
  }
}, {
  "config" : {
    "activeConfig" : {
      "currentSite" : {
        "siteName" : "location1",
        "targetDatabase" : "devl",
        "siteShortName" : "123"
      }
    }
  }
}, {
  "config" : {
    "activeConfig" : {
      "sourceDatabase" : "test",
      "targetSites" : [ {
        "siteName" : "location1",
        "targetDatabase" : "devl",
        "siteShortName" : "123"
      }, {
        "siteName" : "location2",
        "targetDatabase" : "123",
        "siteShortName" : "123"
      } ]
    }
  },
  "secondData" : {
    "queries" : [ {
      "Tablename" : "abc",
      "Query" : "123"
    } ]
  }
}, {
  "config" : {
    "activeConfig" : {
      "currentSite" : {
        "siteName" : "location2",
        "targetDatabase" : "123",
        "siteShortName" : "123"
      }
    }
  }
} ]

此规范创建我正在寻找的结构,但不合并它们。所以我的最终数组最后有4个项目,2个原始文档副本,以及配置数组中的两个项目。我的目标是让配置数组中的这两项合并到文档副本中,因此我有两份原始文档,每份都配置了一个值。

我唯一接近的其他规格是

[
  {
    "operation": "shift",
    "spec": {
      "config": {
        "activeConfig": {
          "targetSites": {
            "*": {
              "@4": "[&]",
              "@": "[&].config.activeConfig.currentSite"
            }
          }
        }
      }
    }
  }
]

这导致最终数组中有两个文档副本,但currentSite部分以每个副本中的配置数组中的所有值结束,而不是每个

1个
[ {
  "config" : {
    "activeConfig" : {
      "sourceDatabase" : "test",
      "targetSites" : [ {
        "siteName" : "location1",
        "targetDatabase" : "devl",
        "siteShortName" : "123"
      }, {
        "siteName" : "location2",
        "targetDatabase" : "123",
        "siteShortName" : "123"
      } ],
      "currentSite" : [ {
        "siteName" : "location1",
        "targetDatabase" : "devl",
        "siteShortName" : "123"
      }, {
        "siteName" : "location2",
        "targetDatabase" : "123",
        "siteShortName" : "123"
      } ]
    }
  },
  "secondData" : {
    "queries" : [ {
      "Tablename" : "abc",
      "Query" : "123"
    } ]
  }
}, {
  "config" : {
    "activeConfig" : {
      "sourceDatabase" : "test",
      "targetSites" : [ {
        "siteName" : "location1",
        "targetDatabase" : "devl",
        "siteShortName" : "123"
      }, {
        "siteName" : "location2",
        "targetDatabase" : "123",
        "siteShortName" : "123"
      } ],
      "currentSite" : [ {
        "siteName" : "location1",
        "targetDatabase" : "devl",
        "siteShortName" : "123"
      }, {
        "siteName" : "location2",
        "targetDatabase" : "123",
        "siteShortName" : "123"
      } ]
    }
  },
  "secondData" : {
    "queries" : [ {
      "Tablename" : "abc",
      "Query" : "123"
    } ]
  }
} ]

(至于为什么,本文档的下一步是将其拆分为NiFi流中的两个流文件,这将允许单独配置每个文件)

感谢您提供的任何输入或帮助。

更新

发现了另一个我很难掌握的有趣行为。

当我使用以下规范时,我得到的输出对我来说没有意义。

规格:

[
  {
    "operation": "shift",
    "spec": {
      "config": {
        "activeConfig": {
          "targetSites": {
            "*": {
              "@4": "[&]",
              "@": "[&].config.activeConfig.currentSite&"
            }
          }
        }
      }
    }
  }
]

输出:

[ {
  "config" : {
    "activeConfig" : {
      "sourceDatabase" : "test",
      "targetSites" : [ {
        "siteName" : "location1",
        "targetDatabase" : "devl",
        "siteShortName" : "123"
      }, {
        "siteName" : "location2",
        "targetDatabase" : "123",
        "siteShortName" : "123"
      } ],
      "currentSite0" : {
        "siteName" : "location1",
        "targetDatabase" : "devl",
        "siteShortName" : "123"
      },
      "currentSite1" : {
        "siteName" : "location2",
        "targetDatabase" : "123",
        "siteShortName" : "123"
      }
    }
  },
  "secondData" : {
    "queries" : [ {
      "Tablename" : "abc",
      "Query" : "123"
    } ]
  }
}, {
  "config" : {
    "activeConfig" : {
      "sourceDatabase" : "test",
      "targetSites" : [ {
        "siteName" : "location1",
        "targetDatabase" : "devl",
        "siteShortName" : "123"
      }, {
        "siteName" : "location2",
        "targetDatabase" : "123",
        "siteShortName" : "123"
      } ],
      "currentSite0" : {
        "siteName" : "location1",
        "targetDatabase" : "devl",
        "siteShortName" : "123"
      },
      "currentSite1" : {
        "siteName" : "location2",
        "targetDatabase" : "123",
        "siteShortName" : "123"
      }
    }
  },
  "secondData" : {
    "queries" : [ {
      "Tablename" : "abc",
      "Query" : "123"
    } ]
  }
} ]

我尝试更改输出路径“@”:“[&] .config.activeConfig.currentSite&”使用&在两个地方。这与我上面的第二个例子的行为类似,其中两个值都在两个副本中结束,但是你可以看到,在这种情况下,一个最终在currentSite0中,一个最终在currentSite1中,在BOTH数组索引0和1中。这意味着& ;当在表达式“[&] .config.activeConfig.currentSite&”中进行评估时,它表现得像是同时具有值​​0和1。我非常清楚地忽略了这种行为的一些细微差别。

1 个答案:

答案 0 :(得分:1)

必须使用两班制。一般来说,在做"东西"对于数组,必须按照""进行移位操作。你正试图这样做。

在您的情况下,您是1)想要将内容复制到输出数组中,2)复制特定的目标网站。

规格

[
  // Step 1: Make the copies of the input data, based on the number
  //  of items in the targetSites array.
  {
    "operation": "shift",
    "spec": {
      "config": {
        "activeConfig": {
          "targetSites": {
            "*": { // targetSites array index
              // go back up 4 levels and grab the whole tree "@4"
              // and write it to the output as a top level array
              // indexed by the "targetSites array index"
              "@4": "[&1]"
            }
          }
        }
      }
    }
  },
  {
    // Step 2 : Annoyingly copy everything across, but use the 
    //  value of the top level array index, to copy the "right" 
    //  data out of the targetSites array.
    "operation": "shift",
    "spec": {
      "*": { // top level array index
        "config": {
          "sourceDatabase": "[&2].config.sourceDatabase", // straight copy across
          "activeConfig": {
            "targetSites": {
              "@": "[&4].config.activeConfig.targetSites", // straight copy across
              //
              // Nifty but very rarely used feature.
              // Use "&3" to lookup the "current" value of the top level array index
              //  and then use that as an index into the targetSites array, and copy
              //  that across as "currentSite"
              "&3": "[&4].config.activeConfig.currentSite"
            }
          }
        },
        "secondData": "[&1].secondData" // straight copy across
      }
    }
  }
]