Question

我正在使用Spark＆amp; amp;尝试使用Intellij中的Kafka斯卡拉。在创建生成器对象时，我无法纠正错误。 Scala对象中的代码如下：

import java.util.Properties 
import org.apache.kafka.clients.producer._ 
import kafka.producer.KeyedMessage 
import org.apache.spark._
object kafkaProducer {

    def main(args: Array[String]){

      val topic = "jovis"

      val props = new Properties()
      props.put("metadata.broker.list", "localhost:9092")
      props.put("serializer.class", "kafka.serializer.StringEncoder")
      val config = new ProducerConfig(props)

//Error in Line below

val producer = new Producer[String, String](config)
      val conf = new SparkConf().setAppName("Kafka").setMaster("local")
      //val ssc = new StreamingContext(conf, Seconds(10))
      val sc = new SparkContext(conf)
      val data = sc.textFile("/home/hdadmin/empname.txt")

      var i = 0
      while(i <= data.count){
        data.collect().foreach(x => {
          println(x)
          producer.send(new KeyedMessage[String, String](topic, x))
          Thread.sleep(1000)
        })
      }

错误日志：

constructor ProducerConfig in class ProducerConfig cannot be accessed in object kafkaProducer
val config = new ProducerConfig(props)

Trait Producer is abstract;Cannot be instantiated.
val producer = new Producer[String, String](config)

我已导入以下依赖关系jar： http://central.maven.org/maven2/org/apache/kafka/kafka-clients/0.8.2.0/kafka-clients-0.8.2.0.jar http://central.maven.org/maven2/org/apache/kafka/kafka_2.11/0.10.2.1/kafka_2.11-0.10.2.1.jar

除此之外，我还启动了zookeeper服务器。

我哪里错了？

Answer 1

可能这会对你有所帮助 what is the difference between kafka ProducerRecord and KeyedMessage

请尝试新的API“org.apache.kafka”%%“kafka”％“0.8.2.0”

import org.apache.kafka.clients.producer.ProducerRecord
import org.apache.kafka.clients.producer.KafkaProducer
val producer = new KafkaProducer[String, String](props)
producer.send(new ProducerRecord[String, String](topic, key, value)

无法在Intellij

1 个答案: