Question

有两个kafka主题：

日志（文本格式），对于这种情况，我使用标准的StringSerializer for Kafka
事件（采用JSON格式），对于本案例，我使用自定义JSON Serializer for Kafka

有一些REST Web应用程序（基于Servlet）。哪种方法最适合此应用程序？

方法1 ：为这两个主题创建单个生产者：

val producer = new KafkaProducer[String, AnyRef](...props...)

// send logs
producer.send(new ProducerRecord[String, AnyRef](
       topic = "logs", "some log key", "some log str"))

// send events
producer.send(new ProducerRecord[String, AnyRef](
       topic = "events", "some evt key", Event("some"))

方法2 ：创建两个具有强类型值的生产者。

val logsProducer = new KafkaProducer[String, String](...props...)
val eventsProducer = new KafkaProducer[String, Event](...props...)

// send logs
logsProducer.send(new ProducerRecord[String, String](
       topic = "logs", "some log key", "some log str"))

// send events
eventsProducer.send(new ProducerRecord[String, Event](
       topic = "events", "some evt key", Event("some event"))

更新1 ：对于Approach-1，我使用基于Json4s的自带序列化器：

class KafkaJson4sSerializer[T <: AnyRef] extends Serializer[T] {

  import org.json4s._
  import org.json4s.native.Serialization
  import org.json4s.native.Serialization.write

  implicit val formats = Serialization.formats(NoTypeHints)

  override def configure(configs: util.Map[String, _], isKey: Boolean): Unit = {}

  override def serialize(topic: String, data: T): Array[Byte] = {

    write(data).getBytes
  }

  override def close(): Unit = {}
}

val p = new Properties()
p.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, 
       "org.apache.kafka.common.serialization.StringSerializer")
p.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, 
       "KafkaJson4sSerializer") // use the above own serializer
val producer = new KafkaProducer[String, AnyRef](p)

// send string type to topic 'logs'
producer.send(new ProducerRecord[String, AnyRef]("logs", "k1", "string value"))
// send Event type to another topic 'events'
producer.send(new ProducerRecord[String, AnyRef]("events", "k2", Event("some evt")))

卡夫卡：两个主题与两个制作人的制作人

0 个答案: