Flink:同步/连接两个流

时间:2016-06-28 10:36:32

标签: scala flink-streaming flink-cep

我有两个流,第一个,带有CEP库,它用于检测模式ON-OFF。在第二个流中是我必须添加的值。我正在寻找连接两个流的方法,这样当我收到一个OFF帧时,我会返回设备状态的变化,其最终值是第二类帧的总和:

Device 311 State Change:Ignition OFF  Value:35.91

我的代码:

// IGNITION pattern
val ig = timedValue.filter(_._2 == "Ignition").keyBy(_._4)
val patternIG_ON: Pattern[(Long,String,String,Long), _] = Pattern.begin[(Long,String,String,Long)]("start").where(_._3 == "OFF").next("end").where(_._3 == "ON")
val patternStream: PatternStream[(Long,String,String,Long)] = CEP.pattern(ig, patternIG_ON)
val patternIG_OFF: Pattern[(Long,String,String,Long), _] = Pattern.begin[(Long,String,String,Long)]("start").where(_._3 == "ON").next("end").where(_._3 == "OFF")
val patternStreamIG_OFF: PatternStream[(Long,String,String,Long)] = CEP.pattern(timedValue, patternIG_OFF)

def selectFn(pattern : mutable.Map[String,(Long,String,String,Long)]): String = {
  val startEvent = pattern.get("start").get
  val endEvent = pattern.get("end").get
  "Device "+startEvent._4 +" State Change: "+startEvent._2+" ON"
}
def selectFnOFF(pattern : mutable.Map[String,(Long,String,String,Long)]): String = {
  val startEvent = pattern.get("start").get
  val endEvent = pattern.get("end").get
  "Device "+startEvent._4 +" State Change: "+startEvent._2+" OFF "
}

val patternStreamSelected = patternStream.select(selectFn(_)).print()
val suma: org.apache.flink.streaming.api.scala.DataStream[(Long, Double)] = odo.map(s => (s._4,s._3)).keyBy(_._1).mapWithState(
  (in, state: Option[(Int, Double)]) => state match {
    case Some((count, sum)) => ((in._1,(sum + in._2).toDouble), Some((count + 1, sum + in._2)))
    case None => ((in._1, in._2), Some(1, in._2))
  })
val patternStreamSelectedOFF = patternStreamIG_OFF.select(selectFnOFF(_)).connect(suma).keyBy(_._1, _._1).map(l => l + " Value:", r => r._2).print()

更新:我的结果:

24.2
23.5
44.010000000000005
67.05000000000001
Device 311 State Change3:Ignition ON
(311,Device 311 State Change: Ignition OFF ) Value:

第一个数字(24.2)来自另一个设备,它不应该出现......

知道什么是最好的方法? 谢谢

0 个答案:

没有答案