在分配水印功能之前使用重新平衡会影响事件时间处理
Configuration config = new Configuration();
StreamExecutionEnvironment see = StreamExecutionEnvironment.createLocalEnvironmentWithWebUI(config);
see.setStreamTimeCharacteristic(TimeCharacteristic.EventTime);
DataStream<String> stream = see
.addSource(new FlinkKafkaConsumer<>("testTopic3", new SimpleStringSchema(), properties));
stream.rebalance()
.map(event->{
JSONObject jsonEvent = new JSONObject(event);
Event formedEvent = new Event();
formedEvent.setTimestamp(Long.parseLong(jsonEvent.getString("time")));
System.out.println("formed event : "+formedEvent.getTimestamp());
return formedEvent;
})
.assignTimestampsAndWatermarks(new TimestampExtractor())
.keyBy(event->{
return "1";
}).window(SlidingEventTimeWindows.of(Time.seconds(5), Time.seconds(5)))
.trigger(new TriggerFunc())
.process(new ProcessWindowFunction());
see.execute();
使用重新平衡时,事件时间触发器不会触发。删除后,它似乎可以正常工作。有什么理由不起作用吗?
以下似乎有效:
Configuration config = new Configuration();
StreamExecutionEnvironment see = StreamExecutionEnvironment.createLocalEnvironmentWithWebUI(config);
see.setStreamTimeCharacteristic(TimeCharacteristic.EventTime);
DataStream<String> stream = see
.addSource(new FlinkKafkaConsumer<>("testTopic3", new SimpleStringSchema(), properties));
stream
.map(event->{
JSONObject jsonEvent = new JSONObject(event);
Event formedEvent = new Event();
formedEvent.setTimestamp(Long.parseLong(jsonEvent.getString("time")));
System.out.println("formed event : "+formedEvent.getTimestamp());
return formedEvent;
})
.assignTimestampsAndWatermarks(new TimestampExtractor())
.rebalance()
.keyBy(event->{
return "1";
}).window(SlidingEventTimeWindows.of(Time.seconds(5), Time.seconds(5)))
.trigger(new TriggerFunc())
.process(new ProcessWindowFunction());
see.execute();
添加我的触发功能。我的触发功能执行不了什么。只是打印一条语句
public class TriggerFunc extends Trigger<Event, TimeWindow>{
private static final long serialVersionUID = 1L;
@Override
public TriggerResult onProcessingTime(long time, TimeWindow window, TriggerContext ctx) throws Exception {
System.out.println("On processing time purging : " + window.getStart());
return TriggerResult.FIRE;
}
@Override
public TriggerResult onEventTime(long time, TimeWindow window, TriggerContext ctx) throws Exception {
System.out.println("On eventTime time");
return TriggerResult.FIRE_AND_PURGE;
}
@Override
public void clear(TimeWindow window, TriggerContext ctx) throws Exception {
}
@Override
public TriggerResult onElement(ThresholdEvent element, long timestamp, TimeWindow window, TriggerContext ctx)
throws Exception {
System.out.println("On element");
return TriggerResult.CONTINUE;
}
}