Question

我正在尝试在4分区主题上使用kafka Highlevelproducer的键控分区我有以下代码：

   var kafka = require('kafka-node'),
      HighLevelProducer = kafka.HighLevelProducer,
      client = new kafka.Client(Host+":"+Port,client_id),
      producer = new HighLevelProducer(client,{partitionerType: 3});
.
.
   theKey = theKey+1;
   if ( theKey > Nb_key ) {
      theKey = 0;
   }
   var payloads = [
        { topic: Topic, key: theKey, messages: JSON.stringify({"hello": "world", "Timestamp": +timestamp}) }
      ];
      producer.send(payloads, function (err, data) {
      });

我检查过邮件的密钥值是否正确，但仍然会将所有邮件发送到分区1。你觉得我在做什么有什么问题吗？

由于

Answer 1

查看代码，我想我理解了这个问题。 baseProducer正在作为字符串对键执行哈希，结果始终相同，因此始终是相同的分区。我编写了自己的自定义分区程序，使用nb分区在密钥上执行简单的模数，它完美地运行。请参阅以下代码：

   var MyPartitioner = function (partitions, key) {
   key = key || '0';
   var index = parseInt(key) % partitions.length;
   return partitions[index];
   };
   var kafka = require('kafka-node'),
      HighLevelProducer = kafka.HighLevelProducer,
      client = new kafka.Client(Host+":"+Port,client_id),
      producer = new HighLevelProducer(client,{requireAcks: 0,partitionerType: 4},MyPartitioner );
      producer.on('ready', function () {
      timer.setInterval(send_data, '',SendIntervalInUsecs+'u');
   });

nodejs和kafka keyed partitioner

1 个答案: