我使用Kafka流来处理实时数据,并且需要对窗口时间数据进行一些汇总操作。
关于聚合操作,我有两个问题。
这是我的代码:
stream = builder.stream("topic");
windowedKStream = stream.map(XXXXX).groupByKey().windowedBy("5mins");
ktable = windowedKStream.aggregate(()->"", new Aggregator(K,V,result));
// my data is stored in 'result' variable, but I can't get it at the end of the 5 mins window.
// I need to send the 'result' to a 3rd service. But I don't know where to temporarily store it and then how to get it.
// below is the code the call a 3rd service, but the code can't be executed(reachable).
// I think it should be executed every 5 mins when thewindows is over. But it isn't.
result = httpclient.execute('result');
答案 0 :(得分:2)
我想可能想做些类似的事情:
ktable.toStream().foreach((k,v) -> httpclient.execute(v));
每次KTable
被更新(禁用缓存)时,更新记录将被发送到下游,并且foreach
将以v
作为当前聚合结果被执行。