如何将火花流数据从kafka放到JSONArray

时间:2018-03-09 07:41:03

标签: java apache-spark apache-kafka spark-streaming

kafka正在生成Spark正在消耗数据的数据。

JavaStreamingContext ssc = new JavaStreamingContext(sc, new Duration(1000));
Map<String, String> kafkaParams = new HashMap<>();
kafkaParams.put("metadata.broker.list", "localhost:9092");
Set<String> topics = Collections.singleton("mytopic");
JavaPairInputDStream<String, String> directKafkaStream = KafkaUtils.createDirectStream(ssc,
                String.class, String.class, StringDecoder.class, StringDecoder.class, kafkaParams, topics);
JSONArray json = new JSONArray();

directKafkaStream.foreachRDD(rdd -> {
    rdd.foreach(record -> json.put(record._2));
});
System.out.println(json.length()); //printing length as zero
ssc.start();
ssc.awaitTermination();

我无法将值放入JSONArray。

    Caused by: java.io.NotSerializableException: org.json.JSONArray

0 个答案:

没有答案