KafkaProducer用于键控和未键控的ProducerRecords

时间:2016-01-29 01:52:06

标签: scala apache-kafka kafka-producer-api

我在Scala中使用0.9 Kafka Java客户端。

scala> val kafkaProducer = new KafkaProducer[String, String](props)

ProducerRecord有几个构造函数,允许您包含或不包含键和/或分区。

scala> val keyedRecord = new ProducerRecord("topic", "key", "value")
scala> kafkaProducer.send(keyedRecord)

应该没有问题。

但是,未加密的ProducerRecord会出现类型错误。

scala> val unkeyedRecord = new ProducerRecord("topic", "value")
res8: org.apache.kafka.clients.producer.ProducerRecord[Nothing,String] =
        ProducerRecord(topic=topic, partition=null, key=null, value=value

scala> kafkaProducer.send(res8)
<console>:17: error: type mismatch;
 found   :   org.apache.kafka.clients.producer.ProducerRecord[Nothing,String]
 required: org.apache.kafka.clients.producer.ProducerRecord[String,String]
 Note: Nothing <: String, but Java-defined class ProducerRecord is invariant in type K.
 You may wish to investigate a wildcard type such as `_ <: String`. (SLS 3.2.10)
   kafkaProducer.send(res8)
                      ^

这是否违反了Kafka的规则,或者它是否是在Scala中使用此Java API而产生的不必要的预防措施?

更重要的是,将密钥和非密钥消息放在同一个Kafka主题中是不是很糟糕?

谢谢

Javadoc:http://kafka.apache.org/090/javadoc/org/apache/kafka/clients/producer/package-summary.html

修改

更改K中参数KafkaProducer的方差可以解决此问题吗?

1 个答案:

答案 0 :(得分:3)

看起来答案在评论中,但要拼写出来,Scala在未明确提供类型时使用类型推断。既然你写了:

val unkeyedRecord = new ProducerRecord("topic", "value")

未提供密钥,并且它变为null,Scala的类型系统推断为Nothing实例。要解决这个问题,请明确声明类型:

val unkeyedRecord = new ProducerRecord[String,String]("topic", "value")