I'm currently doing a Flink project. The main idea of the project is to read a datastream of JSON's (network logs), correlate them, and generate a new JSON, which is a combination of different JSON's information.
At this moment, I'm able to read the JSON's, genereate a KeyedStream (based on the machine that generates the log), and then generate a window stream of 5 seconds.
The next step I want to perform is to use the apply function to the window and combine the information of each JSON. I'm a bit confused of how to do it.
The code I currently have is the following one:
$newarr = array(
'id' => 'Id',
'payment_status' => 'Payment status',
'created_at' => 'Created At');
$columns = $newarr;
The .apply(new generateMetaAlert()) part is complaining with the next error:
The method apply(WindowFunction,R,Tuple,TimeWindow>) in the type WindowedStream,Tuple,TimeWindow> is not applicable for the arguments (MetaAlertGenerator.generateMetaAlert)
Any other code structure proposal distinct that the one I made up?
Thank you in advance for your help
答案 0 :(得分：1)
当您应用/usr/local/lib/node_modules
函数（不使用匿名类）时，自定义keyBy（第3个字段）中的键的类型应为WindowFunction，因为编译器无法确定你的钥匙的类型。此代码编译时没有错误（考虑到我试图用虚拟代码填充空白）：
Tuple但最简单的方法是使用匿名类，以便保持public class Test {
public Test() {
StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime);
DataStream<String> events = env.readTextFile("datastream.log");
DataStream<Tuple2<String, JSONObject>> MetaAlert
= events
.flatMap(new JSONParser())
.keyBy(0)
.timeWindow(Time.seconds(5))
.apply(new GenerateMetaAlert());
}
public class JSONObject {
}
public class JSONParser implements FlatMapFunction<String, Tuple2<String, JSONObject>> {
@Override
public void flatMap(String s, Collector<Tuple2<String, JSONObject>> collector) throws Exception {
}
}
public class GenerateMetaAlert implements WindowFunction<Tuple2<String, JSONObject>, Tuple2<String, JSONObject>, Tuple, TimeWindow> {
@Override
public void apply(Tuple key, TimeWindow timeWindow, Iterable<Tuple2<String, JSONObject>> iterable, Collector<Tuple2<String, JSONObject>> collector) throws Exception {
}
}
}
类型：
String最后，如果你想保留这个课程，但你也希望保持你的关键类型，你可以实现DataStream<Tuple2<String, JSONObject>> MetaAlert
= events
.flatMap(new JSONParser())
.keyBy(0)
.timeWindow(Time.seconds(5))
.apply(new WindowFunction<Tuple2<String, JSONObject>, Tuple2<String, JSONObject>, Tuple, TimeWindow>() {
@Override
public void apply(Tuple tuple, TimeWindow timeWindow, Iterable<Tuple2<String, JSONObject>> iterable, Collector<Tuple2<String, JSONObject>> collector) throws Exception {
// Your code here
}
});
：
KeySelector