Question

我正在使用Azure（https://bitnami.com/stack/kafka/cloud/azure）上来自Bitnami的Kafka群集。它是一个单节点群集，其中一个VM上具有代理，另一个VM上具有Zookeeper。我已启用端口9090和2181。

我有在与Kafka的Confluent .NET Client（v1.0.0）一起在Azure上运行的.Net Core WebApp。 WebApp已配置了生产者和使用者。

我面临的问题是，启动Web应用后不久，使用者由于心跳到期而失败。但是它同时能够接收来自生产者的一些消息。即使我仅启动一个Consumer，由于心跳过期，它也会在SessionTimeoutMs之后自动失败。

我无法理解为何心跳过期（消费者正在接收消息时）。是否在单独的端口上发生？如果我将SessionTimeoutMs设置为300000（最大值），则恰好在5分钟后过期。

服务器日志（当我在WebApp中添加使用者时-请注意心跳将在16秒内到期）：

<plugin>
  <groupId>org.codehaus.mojo</groupId>
  <artifactId>properties-maven-plugin</artifactId>
  <version>1.0-alpha-1</version>
  <executions>
    <execution>
      <phase>generate-resources</phase>
      <goals>
        <goal>read-project-properties</goal>
      </goals>
      <configuration>
        <files>
          <file>C:\test.properties</file>
        </files>
      </configuration>
    </execution>
  </executions>
</plugin>
<plugin>
  <artifactId>maven-resources-plugin</artifactId>
  <executions>
    <execution>
      <id>copy-resources</id>
      <phase>generate-resources</phase>
      <goals>
        <goal>copy-resources</goal>
      </goals>
      <configuration>
        <outputDirectory>src/main/resources/</outputDirectory>
        <resources>
          <resource>
            <directory>${name}</directory>
            <include>test/**</include>
          </resource>
        </resources>
      </configuration>
    </execution>
  </executions>
</plugin>

我的server.properties：

[2019-05-22 17:57:42,302] INFO [GroupCoordinator 1001]: Preparing to rebalance group test-consumer-group-new2 in state PreparingRebalance with old generation 4 (__consumer_offsets-40) (reason: Adding new member rdkafka-03f8db78-4f74-4cb2-9389-bb949c53281c) (kafka.coordinator.group.GroupCoordinator)
[2019-05-22 17:57:42,303] INFO [GroupCoordinator 1001]: Stabilized group test-consumer-group-new2 generation 5 (__consumer_offsets-40) (kafka.coordinator.group.GroupCoordinator)
[2019-05-22 17:57:42,349] INFO [GroupCoordinator 1001]: Assignment received from leader for group test-consumer-group-new2 for generation 5 (kafka.coordinator.group.GroupCoordinator)
[2019-05-22 17:57:58,403] INFO [GroupCoordinator 1001]: Member rdkafka-03f8db78-4f74-4cb2-9389-bb949c53281c in group test-consumer-group-new2 has failed, removing it from the group (kafka.coordinator.group.GroupCoordinator)
[2019-05-22 17:57:58,403] INFO [GroupCoordinator 1001]: Preparing to rebalance group test-consumer-group-new2 in state PreparingRebalance with old generation 5 (__consumer_offsets-40) (reason: removing member rdkafka-03f8db78-4f74-4cb2-9389-bb949c53281c on heartbeat expiration) (kafka.coordinator.group.GroupCoordinator)
[2019-05-22 17:57:58,404] INFO [GroupCoordinator 1001]: Group test-consumer-group-new2 with generation 6 is now empty (__consumer_offsets-40) (kafka.coordinator.group.GroupCoordinator)

使用者的配置：

listeners=PLAINTEXT://:9092
advertised.listeners=PLAINTEXT://40.X.X.X:9092

我也不了解Kafka中的旧/新世代，也找不到相同的文档。谁能说明一下吗？谢谢。

Answer 1

max.poll.records-可能需要减少；

请注意heartbeat.interval.ms-该值必须设置为小于session.timeout.ms，但通常应设置为不大于该值的1/3

卡夫卡消费者无法发送心跳但收到消息

1 个答案: