Question

我试图使用Scala将Kafka-stream Json数据插入我的Cassandra，但不幸的是卡住了。我的代码是： -

val kafkaStream = KafkaUtils.createDirectStream[String, String, StringDecoder, StringDecoder](ssc, kafkaParams, topics)
val records = kafkaStream.map(_._2)
val collection = records.flatMap(_.split(",")).map(s => event(s(0).toString, s(1).toString))
case class event(vehicleid: String, vehicletype: String)
collection.foreachRDD(x => println(x))
collection.saveToCassandra("traffickeyspace", "test", SomeColumns("vehicleid", "vehicletype"))

我得到的错误是： -

not enough arguments for method saveToCassandra: (implicit connector: com.datastax.spark.connector.cql.CassandraConnector, implicit rwf: com.datastax.spark.connector.writer.RowWriterFactory[event])Unit. Unspecified value parameter rwf. kafkatesting.scala  /SparkRedis/src/com/spark/test  line 48 Scala Problem

和其他错误是： -

could not find implicit value for parameter rwf: com.datastax.spark.connector.writer.RowWriterFactory[event]    kafkatesting.scala  /SparkRedis/src/com/spark/test  line 48 Scala Problem

来自制作人的我的JSON记录是： -

{"vehicleId":"3a92516d-58a7-478e-9cff-baafd98764a3","vehicleType":"Small Truck","routeId":"Route-37","longitude":"-95.30818","latitude":"33.265877","timestamp":"2018-03-28 06:21:47","speed":58.0,"fuelLevel":25.0}

Answer 1

您实际上无法在您拥有的情况下声明您的案例类。必须在顶级范围定义案例类才能获得所需的<script src="https://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script> <div class="form-row field_group_copy hide"> <input name="useredu_id[]" type="hidden"> <select name="useredu_qual[]"> <option value="">Select Qualification</option> <option value="BTECH">B.Tech</option> <option value="MTECH">M.Tech</option> <option value="BSC">B.S.C</option> <option value="MSC">M.S.C</option> </select> <textarea name="useredu_detail[]"></textarea> </div> <div class="data-container"> </div>。点击此处了解更多详情：Scala - No TypeTag Available Exception when using case class to try to get TypeTag?

因此，将您的案例类移动到您所在文件的顶级范围。通过这种方式，它可以获取TypeTag，允许它获取TypeTag，从而可以获取其ColumnMapper implicit RowWriterFactor 1}}

火花kafkastream到scala的cassandra

1 个答案: