U-SQL - 从复杂的json对象中提取数据

时间:2016-12-07 13:21:05

标签: json azure-data-lake u-sql

所以我有很多像这样结构的json文件:

{
    "Id": "2551faee-20e5-41e4-a7e6-57bd20b02a22",
    "Timestamp": "2016-12-06T08:09:57.5541438+01:00",
    "EventEntry": {
        "EventId": 1,
        "Payload": [
            "1a3e0c9e-ef69-4c6a-ac8c-9b2de2fbc701",
            "DHS.PlanCare.Business.BusinessLogic.VisionModels.VisionModelServiceWithoutUnitOfWork.FetchVisionModelsForClientOnReferenceDateAsync(System.Int64 clientId, System.DateTime referenceDate, System.Threading.CancellationToken cancellationToken)",
            25,
            "DHS.PlanCare.Business.BusinessLogic.VisionModels.VisionModelServiceWithoutUnitOfWork+<FetchVisionModelsForClientOnReferenceDateAsync>d__11.MoveNext\r\nDHS.PlanCare.Core.Extensions.IQueryableExtensions+<ExecuteAndThrowTaskCancelledWhenRequestedAsync>d__16`1.MoveNext\r\n",
            false,
            "2197, 6-12-2016 0:00:00, System.Threading.CancellationToken"
        ],
        "EventName": "Duration",
        "KeyWordsDescription": "Duration",
        "PayloadSchema": [
            "instanceSessionId",
            "member",
            "durationInMilliseconds",
            "minimalStacktrace",
            "hasFailed",
            "parameters"
        ]
    },
    "Session": {
        "SessionId": "0016e54b-6c4a-48bd-9813-39bb040f7736",
        "EnvironmentId": "C15E535B8D0BD9EF63E39045F1859C98FEDD47F2",
        "OrganisationId": "AC6752D4-883D-42EE-9FEA-F9AE26978E54"
    }
}

如何创建输出

的u-sql查询
Id, 
Timestamp, 
EventEntry.EventId and 
EventEntry.Payload[2] (value 25 in the example below)

我无法弄清楚如何扩展我的查询

@extract =
     EXTRACT 
         Timestamp DateTime
     FROM @"wasb://xxx/2016/12/06/0016e54b-6c4a-48bd-9813-39bb040f7736/yyy/{*}/{*}.json"
     USING new Microsoft.Analytics.Samples.Formats.Json.JsonExtractor();

@res =
    SELECT Timestamp
    FROM @extract;

OUTPUT @res TO "/output/result.csv" USING Outputters.Csv(); 

我见过一些例子:

U- SQL Unable to extract data from JSON file =&gt;这只查询文档的一个级别,我需要来自多个级别的数据。

U-SQL - Extract data from json-array =&gt;这只查询文档的一个级别,我需要来自多个级别的数据。

2 个答案:

答案 0 :(得分:2)

hello一次支持多个JSONPath。

JSONTuple

答案 1 :(得分:1)

您可能需要查看此GIT示例。 https://github.com/Azure/usql/blob/master/Examples/JsonSample/JsonSample/NestedJsonParsing.usql

这需要2个不同的数据元素并将它们组合在一起,就像你有Payload和Payload模式一样。如果您使用&#34; Donut&#34;创建键值对或&#34;蛋糕和面糊&#34;示例您可以将scema与有效负载匹配,并使用交叉应用爆炸功能。