Question

我想使用滑动窗口处理EvenTime中的事件。滑动间隔为24小时，增量为30分钟。问题是下面的代码为每个事件产生48个计算。在我们的案例中，事件按顺序排列，因此我们只需要评估最新窗口。

谢谢，

德扬

public static void processEventsa(
        DataStream<Tuple2<String, MyEvent>> events) throws Exception {

    events.assignTimestampsAndWatermarks(new MyWatermark()).
            keyBy(0).
            timeWindow(Time.hours(windowSizeHour), Time.seconds(windowSlideSeconds)).
            apply(new WindowFunction<Tuple2<String, MyEvent>, Tuple2<String, MyEvent>, Tuple, TimeWindow>() {
                @Override
                public void apply(Tuple key, TimeWindow window, Iterable<Tuple2<String, MyEvent>> input,
                                          Collector<Tuple2<String, MyEvent>> out) throws Exception {

                    for (Tuple2<String, MyEvent> record : input) {



                    }
                }
            });
}

public class MyWatermark implements
        AssignerWithPunctuatedWatermarks<Tuple2<String, MyEvent>> {

    @Override
    public long extractTimestamp(Tuple2<String, MyEvent> event, long previousElementTimestamp) {
        return event.f1.eventTime;
    }

    @Override
    public Watermark checkAndGetNextWatermark(Tuple2<String, MyEvent> event, long previousElementTimestamp) {
        return new Watermark(event.f1.eventTime);
    }
}

final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
    env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime);

Answer 1

问题在于水印。应该使用AssignerWithPeriodicWatermarks

System.Web.Mvc.Html

仅评估基于事件时间的最新窗口滑动窗口

1 个答案: