Question

您好我正在使用Kafka API 0.10.0.1。我创建了主题kartik并有10个分区。从这个生产者代码我得到输出，因为所有10个消息都是10个分区。但问题是每个分区都有10条消息。我想要的是每个分区应该只获得一条消息，即分区应该随机获得1条消息。或者如果我们有50条消息，那么这些消息应该随机分配给这些分区，而不是每个分区都有50条消息。

public class Producerpar {

public static void main(String[] args) throws Exception{

      String topic = "kartik";
      if(topic == null){
         System.out.println("there has to be some topic");
        return;
      }

 Properties props = new Properties();
 props.put("bootstrap.servers", "localhost:9092");
 props.put("acks", "all");
 props.put("retries", 0);
 props.put("batch.size", 16384);
 props.put("linger.ms", 1);
 props.put("buffer.memory", 33554432);
 props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
 props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");



 Producer<String, String> producer = new KafkaProducer<String, String>(props); 

String msg = "Hey there .. How are you doing ?";    
for(int i = 0; i < 10; i++) {

    List<PartitionInfo> infoList = producer.partitionsFor(topic);
    for(PartitionInfo partitioninfo : infoList){ 
    System.out.println(partitioninfo.partition());


    producer.send(new ProducerRecord<String, String>(topic, Integer.toString(i), Integer.toString(i)));


 System.out.println("Message:  " + msg +  " are sent to partitions: " +partitioninfo.partition()  );


 }

}

Answer 1

您可以在发送邮件时使用callback，这将返回RecordMetaData，您可以从中获取邮件的分区ID。

例如：

Producer<String, String> producer = new KafkaProducer<String, String>(props); 

    for(int i = 0; i < 10; i++) {

        ProducerRecord record = new ProducerRecord<String, String>(topic, Integer.toString(i), Integer.toString(i))
            producer.send(record,
                           new Callback() {
                               public void onCompletion(RecordMetadata metadata, Exception e) {
                                   int partitionId = metadata.partition();
                               }
                           });

     }

需要使用kafka随机打印发送消息到10个分区

1 个答案: