我从kafka获取json消息并将其转换为key,pair值。这是我的kafka制作人代码。
val r = scala.util.Random
for( a <- 1 to 100){
var id1=s"ST_${r.nextInt(50)}"
val date = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss")
val d = new Date()
val dateString = date.format(d)
val date1=date.toString
val temp=r.nextInt(90)
val press=r.nextInt(70)
val hum=r.nextInt(30)
var first= new data(id1,dateString,temp,press,hum)
implicit val writes = Json.writes[data]
val jsonString = Json.toJson(first).toString
val record = new ProducerRecord(TOPIC, "key", jsonString)
producer.send(record)
Thread.sleep(5000)
对于我正在使用的火花消费者
val lines = KafkaUtils.createStream(ssc, "localhost:2181", "spark-streaming-consumer-group", Map("test1" -> 5))
val k=lines.map(_._2)
我的输出是
{"id":"ST_44","date":"2018-02-23 23:56:12","temp":76,"pressure":40,"humidity":29})
我想找到所有100个站点的最低温度值和id。我该怎么办?