当我调用SendMessage(msg)时,我总是得到0个分区。我通过命令行

时间:2018-05-09 10:34:29

标签: go apache-kafka sarama

您使用Sarama和Kafka的配置值是什么?

Kafka版本:kafka_2.12-1.1.0.tgz 转版:1.9.1

package kafka

import (
    "flag"
    "fmt"
    "log"
    "strings"

    "github.com/Shopify/sarama"
)

var partition = flag.Int("partition", 12, "The partition to produce to.")

func Start_producer(payload []byte) {
    flag.Parse()
    s := "mydata"
    topic := &s
    // brokers := &[]string{"172.25.33.175:9092,172.25.33.176:9092,172.25.33.177:9092"}
    // brokers := []string{"172.25.33.175:9092,172.25.33.176:9092,172.25.33.177:9092"}

    config := sarama.NewConfig()
    config.Producer.RequiredAcks = sarama.WaitForAll
    config.Producer.Retry.Max = 5
    config.Producer.Return.Successes = true
    producer, err := sarama.NewSyncProducer(strings.Split("172.25.33.175:9092,172.25.33.176:9092,172.25.33.177:9092", ","), config) //default port

    if err != nil {
        log.Println("ERRR")
        panic(err)
    }
    defer func() {
        if err := producer.Close(); err != nil {
            panic(err)
        }
    }()
    msg := &sarama.ProducerMessage{
        Topic:     *topic,
        Value:     sarama.StringEncoder(payload),
        Partition: int32(*partition),
    }
    fmt.Println("XXXX: ", msg.Partition)
    partition, offset, err := producer.SendMessage(msg)
    if err != nil {
        panic(err)
    }
    fmt.Println()
    fmt.Printf("Message is stored in topic(%s)/partition(%d)/offset(%d)\n", *topic, partition, offset)
    fmt.Println("--------------------------------------------------")
    fmt.Println(partition)

}

问题描述

我正在尝试向我的Kafka代理发送消息流,我有3个指定的节点,我希望消息在12个分区中,但是当我调用SendMessage(msg)时,它返回0个分区。这是为什么?为什么我不能在分区中发送消息?

1 个答案:

答案 0 :(得分:0)

您尚未在配置中指定ManualPartitioner,因此它将使用默认值,而不是遵守您在分区字段中告诉它的内容。

添加到您的配置

config.Producer.Partitioner = NewManualPartitioner 

现在应遵守您的分区值

(文件:https://godoc.org/github.com/Shopify/sarama#example-Partitioner--Manual