我正在使用下面的代码从kafka消费数据,并且我想读取带有数据的偏移量,是否仍然可以使用带有scala的spark从kafka读取偏移量和数据。
val kafkaStream = storedOffsets匹配{
case None =>
KafkaUtils.createDirectStream[String, String, StringDecoder, StringDecoder](ssc, kafkaParams, topics)
case Some(fromOffsets) =>
val messageHandler = (mmd: MessageAndMetadata[String, String]) => (mmd.key, mmd.message)
KafkaUtils.createDirectStream[String, String, StringDecoder, StringDecoder, (String, String)](ssc, kafkaParams, fromOffsets, messageHandler)
}