Kafka 1.0：使用java更改特定主题的分区数

Question

Kafka 1.0中的AdminUtils.addPartition调用似乎已更改了签名，如下所示，从2版本开始 - 0.10.2.0 vs 1.0.0

==========================

Kafka 0.10.2.0

/**
  * Add partitions to existing topic with optional replica assignment
  *
  * @param zkUtils Zookeeper utilities
  * @param topic Topic for adding partitions to
  * @param numPartitions Number of partitions to be set
  * @param replicaAssignmentStr Manual replica assignment
  * @param checkBrokerAvailable Ignore checking if assigned replica broker             is available. Only used for testing
   */
   def addPartitions(zkUtils: ZkUtils,
                     topic: String,
                     numPartitions: Int = 1,
                     replicaAssignmentStr: String = "",
                     checkBrokerAvailable: Boolean = true,
                     rackAwareMode: RackAwareMode = RackAwareMode.Enforced)

==========================

Kafka 1.0.0

 /**
   * Add partitions to existing topic with optional replica assignment
   *
   * @param zkUtils Zookeeper utilities
   * @param topic Topic for adding partitions to
   * @param existingAssignment A map from partition id to its assigned      replicas
   * @param allBrokers All brokers in the cluster
   * @param numPartitions Number of partitions to be set
   * @param replicaAssignment Manual replica assignment, or none
   * @param validateOnly If true, validate the parameters without actually      adding the partitions
   * @return the updated replica assignment
   */
   def addPartitions(zkUtils: ZkUtils,
                     topic: String,
                     existingAssignment: Map[Int, Seq[Int]],
                     allBrokers: Seq[BrokerMetadata],
                     numPartitions: Int = 1,
                     replicaAssignment: Option[Map[Int, Seq[Int]]] = None,
                     validateOnly: Boolean = false): Map[Int, Seq[Int]] =

有关如何在Java中轻松使用新的addPartitions调用而无需指定现有副本和新副本分配的任何想法？

Answer 1

试试这个， 它为我工作

Seq<String> names = JavaConverters.asScalaBuffer(Arrays.asList(topicname)).toSeq();
        Seq<BrokerMetadata> brokers = AdminUtils.getBrokerMetadatas(zkUtils, Enforced$.MODULE$, Option.empty());

        Map<String, Map<Object, Seq<Object>>> assignment
            = zkUtils.getPartitionAssignmentForTopics(names);
        Option<Map<Object, Seq<Object>>> assgn = assignment.get(topicname);

        AdminUtils.addPartitions(zkUtils, topicname, assgn.get(), brokers, partitions, Option.empty(), false);

Answer 2

@Munish - 谢谢你的回复。这是适合我的代码。

Seq<String> names = JavaConverters.asScalaBuffer(Arrays.asList(topicName));

Seq<BrokerMetadata> brokers = AdminUtils.getBrokerMetadatas(zkUtils,RackAwareMode.Enforced$.MODULE$, Option.empty());

scala.collection.mutable.Map<String, scala.collection.Map<Object, Seq<Object>>> assignment = (scala.collection.mutable.Map<String,     scala.collection.Map<Object, Seq<Object>>>)zkUtils.getPartitionAssignmentForTopics(names);

Map<String, scala.collection.Map<Object, Seq<Object>>> partitionaAssigmentMap = JavaConverters.mutableMapAsJavaMap(assignment);

AdminUtils.addPartitions(zkUtils, topicName, partitionaAssigmentMap.get(topicName), brokers,partitionCount, Option.empty(), false);

Answer 3

从1.0.0版开始，您可以改用kafka客户程序库

<dependency>
    <groupId>org.apache.kafka</groupId>
    <artifactId>kafka-clients</artifactId>
    <version>1.0.0</version> <!-- or newer -->
</dependency>

在Java代码中，它看起来像：

NewPartition newPartitionRequest = NewPartitions.increaseTo(10);
client.createPartitions(Collections.singletonMap("topicName", newPartitionRequest)).all().get();  //kafka clients AdminClient