如何使用时间窗口实现Flink CEP?

时间:2018-04-16 13:31:42

标签: apache-flink flink-streaming flink-cep

我想使用Flink CEP实现事件警报。

我的用例: 我想将时间窗口应用一小时,如果设备使用率大于某个阈值(100),那么我想提醒最终用户并在特定时间窗口中仅触发一次该警报。

问题:

  1. 按时间窗口,Pattern流在时间窗口完成后获取元素。所以它在实时场景中不起作用。有没有解决方案?
  2. 我可以在时间窗前获取Pattern流中的元素吗?
  3. 如果为最终用户找到匹配的模式,我该如何仅提醒一次?
  4. 的SourceStream:

    KeyedStream<UsageStatistics, Tuple> keyedStreamByMac = kafkaSourceStream.keyBy(UsageStatisticsKey.clusterId.name(), UsageStatisticsKey.deviceMac.name());
    
    SingleOutputStreamOperator<MacTotalUsage> macStream = keyedStreamByMac.timeWindow(Time.hours(1)).reduce(new ReduceFunction<UsageStatistics>() {
                @Override
                public UsageStatistics reduce(UsageStatistics value1, UsageStatistics value2) throws Exception {
                    UsageStatistics statistics = new UsageStatistics();
                    statistics.setDeviceMac(value1.getDeviceMac());
                    statistics.setDownloadBytes(value1.getDownloadBytes() + value2.getDownloadBytes());
                    statistics.setUploadBytes(value1.getUploadBytes() + value2.getUploadBytes());
                    statistics.setClusterId(value1.getClusterId());
                    return statistics;
                }
            }).map(new MapFunction<UsageStatistics, MacTotalUsage>() {
    
                @Override
                public MacTotalUsage map(UsageStatistics value) {
                    MacTotalUsage macTotalUsage = new MacTotalUsage();
                    macTotalUsage.setClusterId(value.getClusterId());
                    macTotalUsage.setDeviceMac(value.getDeviceMac());
                    macTotalUsage.setTotalDownloadBytes(value.getDownloadBytes());
                    macTotalUsage.setTotalUploadBytes(value.getUploadBytes());
                    return macTotalUsage;
                }
            });
    

    模式:

    Pattern<MacTotalUsage, MacTotalUsage> maxUsagePattern = Pattern.<MacTotalUsage>begin("first").where(new SimpleCondition<MacTotalUsage>() {
                @Override
                public boolean filter(MacTotalUsage event) throws Exception {
                    return (event.getTotalDownloadBytes() >= 100);
                }
            });
    

    PatternStream:

    DataStreamOperator<MaxUsageAlert> maxUsageAlert = CEP.pattern(macStream, maxUsagePattern).select(new PatternSelectFunction<MacTotalUsage, MaxUsageAlert>() {
        @Override
        public MaxUsageAlert select(Map<String, List<MacTotalUsage>> pattern) throws Exception {
            MacTotalUsage event = pattern.get("first").get(0);
            return new MaxUsageAlert(event);
        }
    });
    

    如果有人提供解决方案,那将非常有用。

0 个答案:

没有答案