In stream analytics query how to order the data based on column before sending it to a UDA function

时间:2019-01-07 13:54:46

标签: azure-stream-analytics

So I have an azure stream analytics query. It is joining 2 inputs and that needs to be the input of a user defined aggregate function. In the UDA function, I need to do some string concatenation of the incoming rows and output one final string. So the order or the rows from the join is important to be in order of one of the columns which is a string. SA jobs do not allow me to order, so how can I accomplish this. Below is the code extract of the query..

CalcData AS
(
SELECT 
  x.fqn AS fqn,       
  x.value as xvalue,
  y.value as yvalue,
  x.time as time
FROM (select fqn
,value, time from DataInput1 ) y
join
(SELECT
fqn as fqn,
 value as value,
  time as time
FROM DataInput2 ) x on y.time=x.time and x.fqn=y.fqn and DATEDIFF(second, x, y) = 0
--order by time asc, fqn 
),
FormatData AS
(
SELECT UDA.svgstring(CalcData) AS v,
time    FROM CalcData
GROUP BY time,TumblingWindow(minute, 1)
)

I need to order by column fqn, but it gives the error

Syntax errors: The ORDER BY clause is not valid in views, inline functions, derived tables, sub-queries, and common table expressions, unless TOP or FOR XML or OFFSET is also specified.

Any help will be greatly appreciated.

Edit: also I tried

I tried

CalcData AS
 (
SELECT TOP 10000
x.fqn AS fqn,       
x.value as xvalue,
y.value as yvalue,
x.time as time
FROM (select fqn
,value, time from DataInput1 ) y
join
(SELECT
fqn as fqn,
value as value,
time as time
FROM DataInput2 ) x on y.time=x.time and x.fqn=y.fqn and DATEDIFF (second,   x, y) = 0
order by time, fqn 
),
FormatData AS
(
SELECT UDA.svgstring(CalcData) AS v,
time    FROM CalcData
 GROUP BY time,TumblingWindow(minute, 1)
)

but it doesn't look like the order by is taking effect

1 个答案:

答案 0 :(得分:1)

Stream Analytics目前没有order by子句。在您最初的查询顺序中,group by之前的顺序为ASA,因为ASA正在处理无限制的流,为了能够按非时间戳列进行排序,因此必须首先将事件分组到窗口中。在该窗口中,您可以订购所有事件。

我看到您有time, fqn按列排序。假设time是有效负载的时间戳,则可以使用select ... from input timestamp by time按应用程序时间进行排序。

然后您可以在UDA.svgstring内进行另一级订购。该函数的输入将不按fqn进行排序,而是按time进行排序。您将必须累积所有事件,并在调用computeResult()时,按fqn进行排序,并在有序行上创建字符串。