Question

我正在尝试使用Apache Flink创建一个滑动窗口。我使用SourceFunction发送一些数据但是，它确实通过读取数据来满足我的条件。当我在控制台上读取输出时数据被转换为序列化值时出现问题。我希望它以常规POJO形式显示。

这是我的数据生成器代码：

public class DataCollector implements SourceFunction<MachineData> {
@Override
public void run(SourceContext<MachineData> sourceContext) throws Exception {
    DateFormat df = new SimpleDateFormat ("yyyy-MM-dd");
    MachineData m1 = new MachineData (101,"Eng1",df.parse("2016-04-01"), 90);
    MachineData m2 = new MachineData (102,"Eng1",df.parse("2016-04-01"), 105);

    sourceContext.collect (m1);
    sourceContext.collect (m2);
}

@Override
public void cancel() {

}}

这是我的窗口代码：

public class AppWindow {

public static void main(String[] args) throws Exception {
    StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment ();
    env.registerType (MachineData.class);

    env.setParallelism (2);
    env.enableCheckpointing (1500, CheckpointingMode.EXACTLY_ONCE);
    env.setStreamTimeCharacteristic (TimeCharacteristic.ProcessingTime);


    DataStream<MachineData> stream = env.addSource (new DataCollector ());

    stream
            .filter (reading -> reading.getvTemp ()>100)
            .map (reading -> "-- ALERT Reading Above Threshold!: " + reading)
            .print ();

    stream
            .keyBy (reading -> reading.getEngineId ())
            .window (SlidingTimeWindows.of (Time.seconds (10),Time.seconds (10)));

    env.execute ();

}}

此外，我希望它能够显示常规的vTemp读数，这些读数不高于阈值，我该怎么做？

输出我得到：

    INFO | Filter -> Map -> Sink: Unnamed (2/2) (bc7ddef1a04a90951c210c3e4879073f) switched from DEPLOYING to RUNNING
 INFO | 05/31/2018 17:00:57 Filter -> Map -> Sink: Unnamed(1/2) switched to RUNNING 
05/31/2018 17:00:57 Filter -> Map -> Sink: Unnamed(1/2) switched to RUNNING 
 INFO | 05/31/2018 17:00:57 Filter -> Map -> Sink: Unnamed(2/2) switched to RUNNING 
05/31/2018 17:00:57 Filter -> Map -> Sink: Unnamed(2/2) switched to RUNNING 
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.esotericsoftware.kryo.util.UnsafeUtil (file:/root/.m2/repository/com/esotericsoftware/kryo/kryo/2.24.0/kryo-2.24.0.jar) to constructor java.nio.DirectByteBuffer(long,int,java.lang.Object)
WARNING: Please consider reporting this to the maintainers of com.esotericsoftware.kryo.util.UnsafeUtil
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
 INFO | Source: Custom Source (1/1) switched to FINISHED
 INFO | Freeing task resources for Source: Custom Source (1/1)
 INFO | Unregistering task and sending final execution state FINISHED to JobManager for task Source: Custom Source (50a21edf25560021864b483211196b58)
 INFO | Source: Custom Source (1/1) (50a21edf25560021864b483211196b58) switched from RUNNING to FINISHED
 INFO | 05/31/2018 17:00:58 Source: Custom Source(1/1) switched to FINISHED 
05/31/2018 17:00:58 Source: Custom Source(1/1) switched to FINISHED 
**2> -- ALERT Reading Above Threshold!: fkwtest.MachineData@577aa2db**
 INFO | Filter -> Map -> Sink: Unnamed (2/2) switched to FINISHED
 INFO | Freeing task resources for Filter -> Map -> Sink: Unnamed (2/2)
 INFO | Unregistering task and sending final execution state FINISHED to JobManager for task Filter -> Map -> Sink: Unnamed (bc7ddef1a04a90951c210c3e4879073f)
 INFO | Filter -> Map -> Sink: Unnamed (2/2) (bc7ddef1a04a90951c210c3e4879073f) switched from RUNNING to FINISHED
 INFO | 05/31/2018 17:00:58 Filter -> Map -> Sink: Unnamed(2/2) switched to FINISHED 
05/31/2018 17:00:58 Filter -> Map -> Sink: Unnamed(2/2) switched to FINISHED

任何帮助将不胜感激。

Flink Source Function返回序列化值

0 个答案: