是否可以在flink流中知道本地窗口属于哪个子任务?我想在getRuntimeContext().getIndexOfThisSubtask()
实现中使用TriggerPolicy
方法。
答案 0 :(得分:2)
目前无法获得运行TriggerPolicy
的窗口化运算符的子任务索引。
但是,您可以通过在上游放置map
操作来解决这个问题,该操作为每个数据元素分配子任务的当前索引。
DataStream<Tuple2<Integer, String>> ds = env.fromElements(
new Tuple2<Integer, String>(1, "a"),
new Tuple2<Integer, String>(2, "b"),
new Tuple2<Integer, String>(1, "c"),
new Tuple2<Integer, String>(2, "d"));
ds.groupBy(0)
.map(new RichMapFunction<Tuple2<Integer,String>, Tuple3<Integer, Integer, String>>() {
@Override
public Tuple3<Integer, Integer, String> map(Tuple2<Integer, String> integerStringTuple2) throws Exception {
return new Tuple3<Integer, Integer, String>(
getRuntimeContext().getIndexOfThisSubtask(),
integerStringTuple2.f0,
integerStringTuple2.f1);
}
})
.window(new TestingTriggerPolicy(), new TestingEvictionPolicy())
.mapWindow(new WindowMapFunction<Tuple3<Integer, Integer, String>, String>() {
@Override
public void mapWindow(Iterable<Tuple3<Integer, Integer, String>> iterable, Collector<String> collector) throws Exception {
StringBuilder builder = new StringBuilder();
for (Tuple3<Integer, Integer, String> element : iterable) {
builder.append(element.toString() +"; ");
}
collector.collect(builder.toString());
}
})