火花流Dstream订户到代理

时间:2018-09-02 21:06:53

标签: spark-streaming publish-subscribe broker subscriber

我编写了一个火花流应用程序,该应用程序使用存储在pubsub主题(请求订阅者)中的数据。 在spark作业运行时期间,我得到以下日志信息:

   18/09/02 18:57:19 INFO com.renat.det.pubsub.connector.PubsubInputDStream: Removing blocks of RDD WriteAheadLogBackedBlockRDD[1886] at ReceiverInputDStream at PubsubInputDStream.scala:27 of time 1535913504000 ms
18/09/02 18:57:32 INFO com.renat.det.pubsub.connector.PubsubInputDStream: Removing blocks of RDD WriteAheadLogBackedBlockRDD[1895] at ReceiverInputDStream at PubsubInputDStream.scala:27 of time 1535913516000 ms
18/09/02 18:57:50 INFO com.renat.det.pubsub.connector.PubsubInputDStream: Removing blocks of RDD WriteAheadLogBackedBlockRDD[1898] at ReceiverInputDStream at PubsubInputDStream.scala:27 of time 1535913528000 ms
18/09/02 18:58:07 INFO com.renat.det.pubsub.connector.PubsubInputDStream: Removing blocks of RDD WriteAheadLogBackedBlockRDD[1913] at ReceiverInputDStream at PubsubInputDStream.scala:27 of time 1535913540000 ms
18/09/02 18:58:10 INFO com.renat.det.pubsub.connector.PubsubInputDStream: Removing blocks of RDD WriteAheadLogBackedBlockRDD[1916] at ReceiverInputDStream at PubsubInputDStream.scala:27 of time 1535913552000 ms
18/09/02 18:58:22 INFO com.renat.det.pubsub.connector.PubsubInputDStream: Removing blocks of RDD WriteAheadLogBackedBlockRDD[1925] at ReceiverInputDStream at PubsubInputDStream.scala:27 of time 1535913564000 ms
18/09/02 18:58:39 INFO com.renat.det.pubsub.connector.PubsubInputDStream: Removing blocks of RDD WriteAheadLogBackedBlockRDD[1928] at ReceiverInputDStream at PubsubInputDStream.scala:27 of time 1535913576000 ms
18/09/02 18:58:51 INFO com.renat.det.pubsub.connector.PubsubInputDStream: Removing blocks of RDD WriteAheadLogBackedBlockRDD[1937] at ReceiverInputDStream at PubsubInputDStream.scala:27 of time 1535913588000 ms
18/09/02 18:59:03 INFO com.renat.det.pubsub.connector.PubsubInputDStream: Removing blocks of RDD WriteAheadLogBackedBlockRDD[1946] at ReceiverInputDStream at PubsubInputDStream.scala:27 of time 1535913600000 ms
18/09/02 18:59:21 INFO com.renat.det.pubsub.connector.PubsubInputDStream: Removing blocks of RDD WriteAheadLogBackedBlockRDD[1955] at ReceiverInputDStream at PubsubInputDStream.scala:27 of time 1535913612000 ms
18/09/02 18:59:39 INFO com.renat.det.pubsub.connector.PubsubInputDStream: Removing blocks of RDD WriteAheadLogBackedBlockRDD[1960] at ReceiverInputDStream at PubsubInputDStream.scala:27 of time 1535913624000 ms
18/09/02 18:59:52 INFO com.renat.det.pubsub.connector.PubsubInputDStream: Removing blocks of RDD WriteAheadLogBackedBlockRDD[1967] at ReceiverInputDStream at PubsubInputDStream.scala:27 of time 1535913636000 ms
18/09/02 19:00:26 INFO com.renat.det.pubsub.connector.PubsubInputDStream: Removing blocks of RDD WriteAheadLogBackedBlockRDD[1976] at ReceiverInputDStream at PubsubInputDStream.scala:27 of time 1535913648000 ms
18/09/02 19:00:45 INFO com.renat.det.pubsub.connector.PubsubInputDStream: Removing blocks of RDD WriteAheadLogBackedBlockRDD[1979] at ReceiverInputDStream at PubsubInputDStream.scala:27 of time 153

该工作已成功执行,我得到了预期的输出。我只想了解这些日志信息的含义。

0 个答案:

没有答案