流分析 - 将传入的JSON转换为PowerBI流数据集

时间:2018-05-15 13:46:47

标签: azure azure-stream-analytics stream-analytics

我有一个传感器,它以下列格式(通过Modbus服务器)向IoTHub报告数据:

15/05/2018 14:56:56> Device: [dev], Data:[[{"DisplayName":"Temperature","HwId":"PI-1","Address":"400002","Value":"192","SourceTimestamp":"2018-05-15 13:56:52"},{"DisplayName":"Humidity","HwId":"PI-1","Address":"400001","Value":"397","SourceTimestamp":"2018-05-15 13:56:52"}]]Properties:
'content-type': 'application/edge-modbus-json'

15/05/2018 14:57:00> Device: [dev], Data:[[{"DisplayName":"Temperature","HwId":"PI-1","Address":"400002","Value":"201","SourceTimestamp":"2018-05-15 13:56:57"},{"DisplayName":"Humidity","HwId":"PI-1","Address":"400001","Value":"397","SourceTimestamp":"2018-05-15 13:56:57"}]]Properties:
'content-type': 'application/edge-modbus-json'

15/05/2018 14:57:06> Device: [dev], Data:[[{"DisplayName":"Temperature","HwId":"PI-1","Address":"400002","Value":"201","SourceTimestamp":"2018-05-15 13:57:02"},{"DisplayName":"Humidity","HwId":"PI-1","Address":"400001","Value":"397","SourceTimestamp":"2018-05-15 13:57:02"}]]Properties:
'content-type': 'application/edge-modbus-json'

15/05/2018 14:57:10> Device: [dev], Data:[[{"DisplayName":"Temperature","HwId":"PI-1","Address":"400002","Value":"195","SourceTimestamp":"2018-05-15 13:57:07"},{"DisplayName":"Humidity","HwId":"PI-1","Address":"400001","Value":"397","SourceTimestamp":"2018-05-15 13:57:07"}]]Properties:
'content-type': 'application/edge-modbus-json'

每个传感器都在一个单独的数组条目中报告,并且还在传感器的Displayname和Value之间进行分割。

我想要的是一个JSON Payload,我可以将其输入PowerBI,因此需要按顺序排列:

时间戳:时间,湿度:humidity_value,温度:temperature_value

如何构建合适的流分析查询来执行此操作?这种输入格式是典型的Modbus或OPC-UA型设备,因此很可能会遇到这种情况。

我尝试使用GetArrayElement /(s)但是数组在JSON中没有名称,所以没有什么可以引用。

2 个答案:

答案 0 :(得分:0)

根据我的理解,您可以利用Azure Stream Analytics JavaScript user-defined functions来平衡数据。

假设您的数据如下所示:

{
    Device:"dev01", 
    Data:[
    [
      {"DisplayName":"Temperature","HwId":"PI-1","Address":"400002","Value":"192","SourceTimestamp":"2018-05-15 13:56:52"}
      ,{"DisplayName":"Humidity","HwId":"PI-1","Address":"400001","Value":"397","SourceTimestamp":"2018-05-15 13:56:52"}
     ]
    ]
}   

您可以创建以下UDF:

<强> GetValueByPropertyName:

function main(arrs,propertyname) {
    for(var i=0;i<arrs.length;i++){
       var item=arrs[i];
       if(item.hasOwnProperty(propertyname)){
           return item[propertyname];
       }
    }
    return '';
}

<强> GetValueByDisplayName:

function main(dataArr,displayname) {
    if(dataArr){
      for(var j=0;j<dataArr.length;j++)
      {
        var subArr=dataArr[j];
        for(var i=0;i<subArr.length;i++){
          var obj=subArr[i];
          if(obj.DisplayName.toLowerCase()==displayname.toLowerCase())
              return obj.Value;
        }
      }
    }
    return '';
}

示例查询:

select 
input.device as DeviceName,
UDF.GetValueByDisplayName(input.Data,'Temperature') as Temperature,
UDF.GetValueByDisplayName(input.Data,'Humidity') as Humidity,
UDF.GetValueByPropertyName(GetArrayElement(input.Data,0),'SourceTimestamp') as [Timestamp]
from input

<强> TEST:

enter image description here

答案 1 :(得分:0)

实际上找到了一种更简单的方法来直接处理这个没有功能。将CASE语句与LAST结合使用。

SELECT System.Timestamp as timestamp,

    CASE Address
        WHEN '400001' THEN cast(Value as float)/10 ELSE last(cast(Value as float)/10) over (partition by HwId limit duration(day,1) when Value is not null and Address like '400001')
    END
    AS Humidity,
    CASE Address
        WHEN '400002' THEN cast(Value as float)/10 ELSE last(cast(Value as float)/10) over (partition by HwId limit duration(day,1) when Value is not null and Address like '400002')
    END
    AS Temperature,
    CASE Address
        WHEN '400003' THEN cast(Value as float)/10 ELSE last(cast(Value as float)/10) over (partition by HwId limit duration(day,1) when Value is not null and Address like '400003')
    END
    AS Pressure    
INTO PowerBI
FROM IoTHub as event