M语言多个json查询总结果

Question

我有一个rest api查询，它返回最多500条记录。我也可以用api获取记录总数。

因此，我需要合并多个查询，以获取我的所有记录。此外，我需要查询是动态的，并始终与总记录保持同步。

这是最多1000条记录的示例代码：

let
    Source = Json.Document(Web.Contents("https://example.pipedrive.com/v1/deal:(id,title,value,currency)?api_token=12345&limit=500&start=0")),
    Source2 = Json.Document(Web.Contents("https://example.pipedrive.com/v1/deal:(id,title,value,currency)?api_token=12345&limit=500&start=500")),
    data = Source[data],
    data2 = Source2[data],
    #"Converted to Table" = Table.FromList(data, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
    #"Converted to Table 2" = Table.FromList(data2, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
    #"Expanded Column1" = Table.ExpandRecordColumn(#"Converted to Table", "Column1", {"id", "title", "value", "currency"}, {"Column1.id", "Column1.title", "Column1.value", "Column1.currency"}),
    #"Expanded Column2" = Table.ExpandRecordColumn(#"Converted to Table", "Column1", {"id", "title", "value", "currency"}, {"Column1.id", "Column1.title", "Column1.value", "Column1.currency"}),
    #"Appended Query" = Table.Combine({#"Expanded Column1", #"Expanded Column2"})
in
    #"Expanded Column1"

我的总记录大约是11000，每周都在变化。

我可以使用data.additional_data.total_records获取记录总数。

我可以在变量上使用此值，并使用某种程度的总计查询（总记录数/ 500）吗？

任何想法？

Answer 1

您可以参数化查询并将其转换为函数，而不是逐个获取数据并合并查询。

让我们假设您的原始查询如下：

let
    Source = Json.Document(Web.Contents("https://example.pipedrive.com/v1/deal:(id,title,value,currency)?api_token=12345&limit=500&start=0")),
    data = Source[data],
    #"Converted to Table" = Table.FromList(data, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
    #"Expanded Column1" = Table.ExpandRecordColumn(#"Converted to Table", "Column1", {"id", "title", "value", "currency"}, {"Column1.id", "Column1.title", "Column1.value", "Column1.currency"})
in
    #"Expanded Column1"

您可以参数化start参数并将查询更改为GetDeal函数：

(start as text) =>
let
    Source = Json.Document(Web.Contents("https://example.pipedrive.com/v1/deal:(id,title,value,currency)?api_token=12345&limit=500&start=" & start)),
    data = Source[data],
    #"Converted to Table" = Table.FromList(data, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
    #"Expanded Column1" = Table.ExpandRecordColumn(#"Converted to Table", "Column1", {"id", "title", "value", "currency"}, {"Column1.id", "Column1.title", "Column1.value", "Column1.currency"})
in
    #"Expanded Column1"

假设您通过导航到total_records将data.additional_data.total_records存储为来自其他查询的值：

您现在可以创建新查询并生成从0到total_records的列表，其增量为500：

let
    Source = List.Generate(() => 0, each _ <= (total_records), each _ + 500)
in
    Source

将其转换为表格并将列更改为text：

let
    Source = List.Generate(() => 0, each _ <= (total_records), each _ + 500),
    #"Converted to Table" = Table.FromList(Source, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
    #"Renamed Columns" = Table.RenameColumns(#"Converted to Table",{{"Column1", "start"}}),
    #"Changed Type" = Table.TransformColumnTypes(#"Renamed Columns",{{"start", type text}})
in
    #"Changed Type"

现在我们可以使用列GetDeal调用自定义函数start：

您应该有一个表列表作为新列，并可以从那里工作。您可以删除start列，因为它不再需要了。