Question

我们将同步pykafka生产者与kafka集群的3个节点一起使用。我们正在尝试替换集群中的代理，以进行一些升级。

在Kafka生产者已经运行的同时，我们已将特定主题的分区重新分配给其他2个Kafka经纪人。分区重新分配成功，但是由于它试图与不再充当该主题/分区生产者的代理进行通信，因此出现了以下错误。

这一直持续到重新启动应用程序服务器为止，即；重新创建KafkaClient对象。

代码：

  def send(self, topic_name: str, data, raise_exception=False) -> None:
        """Publishes a message on a topic
        """
        message = self.encode(data)
        topic = self.client.topics[topic_name.encode('utf-8')]
        try:
            with topic.get_sync_producer(linger_ms=0) as producer:
                producer.produce(message)
        except Exception as e:
            monitoring.record_exception()
            logger.exception('Pushing entry to kafka failed for topic: {topic} and message: {message}'.format(
                topic=topic.name,
                message=message
            ))
            if raise_exception:
                raise e

实际行为：

[WARNING] [2018-12-20 11:13:33,482] [pykafka.producer] [producer.py:520] Produce request for b'test-load-2001'/2 to b'172.16.45.154':9092 failed with error code 3.
[WARNING] [2018-12-20 11:13:33,607] [pykafka.producer] [producer.py:520] Produce request for b'test-load-2001'/2 to b'172.16.45.154':9092 failed with error code 3.
[WARNING] [2018-12-20 11:13:33,778] [pykafka.producer] [producer.py:520] Produce request for b'test-load-2001'/2 to b'172.16.45.154':9092 failed with error code 3.
[WARNING] [2018-12-20 11:13:33,907] [pykafka.producer] [producer.py:520] Produce request for b'test-load-2001'/2 to b'172.16.45.154':9092 failed with error code 3.
[ERROR] [2018-12-20 11:13:34,092] [pykafka.producer] [producer.py:551] Message not delivered!! UnknownTopicOrPartition("Produce request for b'test-load-2001'/2 to b'172.16.45.154':9092 failed with error code 3.",)
[ERROR] [2018-12-20 11:13:34,095] [root] [<console>:8] Failure.
Traceback (most recent call last):
  File "<console>", line 5, in <module>
  File "/usr/local/lib/python3.6/site-packages/pykafka/producer.py", line 413, in produce
    raise exc
pykafka.exceptions.UnknownTopicOrPartition: Produce request for b'test-load-2001'/2 to b'172.16.45.154':9092 failed with error code 3.

预期的行为：Kafka客户端应通过更新群集元数据来自动从这些错误中恢复。

pykafka.exceptions.UnknownTopicOrPartition卡夫卡主题分区重新分配后

0 个答案: