使用flink消费者无法从kinesis中获得任何回报

时间:2018-09-07 18:52:29

标签: apache-flink amazon-kinesis

这是Java代码,使用flink 1.4,在intelij上对其进行调试

  StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();

    env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime);

    Properties kinesisConsumerConfig = new Properties();
    kinesisConsumerConfig.setProperty(AWSConfigConstants.AWS_REGION, "us-west-2");
    kinesisConsumerConfig.setProperty(AWSConfigConstants.AWS_ACCESS_KEY_ID, "");
    kinesisConsumerConfig.setProperty(AWSConfigConstants.AWS_SECRET_ACCESS_KEY, "/Y");
    kinesisConsumerConfig.setProperty(ConsumerConfigConstants.SHARD_GETRECORDS_MAX, "4");
    kinesisConsumerConfig.setProperty(ConsumerConfigConstants.SHARD_GETRECORDS_INTERVAL_MILLIS, "2000");
    kinesisConsumerConfig.setProperty(ConsumerConfigConstants.SHARD_DISCOVERY_INTERVAL_MILLIS, "2000");
    kinesisConsumerConfig.setProperty(ConsumerConfigConstants.SHARD_GETRECORDS_RETRIES, "5");
    kinesisConsumerConfig.setProperty(ConsumerConfigConstants.STREAM_INITIAL_POSITION, "LATEST");

    DataStream<Event> kinesisStream = env.addSource(new FlinkKinesisConsumer<>(
            "ConversionScore",
            new EventSchema(),
            kinesisConsumerConfig)
    );

    DataStream<String> pickupCounts = kinesisStream
            .map(trip -> Long.toString(trip.getTimestamp()))
            .timeWindowAll(Time.seconds(3))
            .apply(new AllWindowFunction<String, String, TimeWindow>() {
              @Override
              public void apply(TimeWindow window, Iterable<String> values, Collector<String> out) throws Exception {
                StringBuilder stringBuilder = new StringBuilder();
                for(String i : values)  {
                  stringBuilder.append(i);
                }
                out.collect(stringBuilder.toString());
              }
            });

    pickupCounts.print();

但是我什么也没回来,Kinesis上面有数据,它的多线程进程,如果有异常显示,我将无法看到我的假设

0 个答案:

没有答案