我遇到这个问题,我在这里说
我有3个代理,复制因子为1。
在挂起的消费者中,我得到以下日志,该日志随着时间的推移几乎没有进展,最后一行相同:
15:01:45.255 [default-akka.kafka.default-dispatcher-6] INFO o.a.k.c.consumer.ConsumerConfig - ConsumerConfig values:
auto.commit.interval.ms = 5000
auto.offset.reset = earliest
bootstrap.servers = [dev-cp-kafka-headless:9092]
check.crcs = true
client.dns.lookup = default
client.id =
connections.max.idle.ms = 540000
default.api.timeout.ms = 60000
enable.auto.commit = false
exclude.internal.topics = true
fetch.max.bytes = 52428800
fetch.max.wait.ms = 10000
fetch.min.bytes = 1
group.id = elsevier-triple-store-builder-26
heartbeat.interval.ms = 60000
interceptor.classes = []
internal.leave.group.on.close = true
isolation.level = read_uncommitted
key.deserializer = class org.apache.kafka.common.serialization.StringDeserializer
max.partition.fetch.bytes = 1048576
max.poll.interval.ms = 5000000
max.poll.records = 10000
metadata.max.age.ms = 300000
metric.reporters = []
metrics.num.samples = 2
metrics.recording.level = INFO
metrics.sample.window.ms = 30000
partition.assignment.strategy = [class org.apache.kafka.clients.consumer.RangeAssignor]
receive.buffer.bytes = 65536
reconnect.backoff.max.ms = 1000
reconnect.backoff.ms = 50
request.timeout.ms = 30000
retry.backoff.ms = 100
sasl.client.callback.handler.class = null
sasl.jaas.config = null
sasl.kerberos.kinit.cmd = /usr/bin/kinit
sasl.kerberos.min.time.before.relogin = 60000
sasl.kerberos.service.name = null
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.login.callback.handler.class = null
sasl.login.class = null
sasl.login.refresh.buffer.seconds = 300
sasl.login.refresh.min.period.seconds = 60
sasl.login.refresh.window.factor = 0.8
sasl.login.refresh.window.jitter = 0.05
sasl.mechanism = GSSAPI
security.protocol = PLAINTEXT
send.buffer.bytes = 131072
session.timeout.ms = 300000
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
ssl.endpoint.identification.algorithm = https
ssl.key.password = null
ssl.keymanager.algorithm = SunX509
ssl.keystore.location = null
ssl.keystore.password = null
ssl.keystore.type = JKS
ssl.protocol = TLS
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.location = null
ssl.truststore.password = null
ssl.truststore.type = JKS
value.deserializer = class org.apache.kafka.common.serialization.StringDeserializer
15:01:45.397 [default-akka.kafka.default-dispatcher-6] INFO o.a.kafka.common.utils.AppInfoParser - Kafka version : 2.1.1
15:01:45.397 [default-akka.kafka.default-dispatcher-6] INFO o.a.kafka.common.utils.AppInfoParser - Kafka commitId : 21234bee31165527
15:01:46.133 [default-akka.kafka.default-dispatcher-22] INFO org.apache.kafka.clients.Metadata - Cluster ID: cLqnPKhKRPuZtkhAFpcaow
15:01:46.201 [default-akka.kafka.default-dispatcher-8] INFO o.a.k.c.c.i.AbstractCoordinator - [Consumer clientId=consumer-1, groupId=elsevier-triple-store-builder-26] Discovered group coordinator dev-cp-kafka-2.dev-cp-kafka-headless.dev:9092 (id: 2147483645 rack: null)
15:01:46.872 [default-akka.kafka.default-dispatcher-10] INFO o.a.k.c.c.i.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=elsevier-triple-store-builder-26] Revoking previously assigned partitions []
15:01:46.874 [default-akka.kafka.default-dispatcher-10] INFO o.a.k.c.c.i.AbstractCoordinator - [Consumer clientId=consumer-1, groupId=elsevier-triple-store-builder-26] (Re-)joining group
15:06:35.500 [default-akka.kafka.default-dispatcher-14] INFO o.a.k.c.c.i.AbstractCoordinator - [Consumer clientId=consumer-1, groupId=elsevier-triple-store-builder-26] Successfully joined group with generation 339
15:06:35.641 [default-akka.kafka.default-dispatcher-19] INFO o.a.k.c.c.i.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=elsevier-triple-store-builder-26] Setting newly assigned partitions [elsevier-proxy-reaxys-document-2, elsevier-proxy-reaxys-protein-4, elsevier-proxy-reaxys-bioactivity-0, elsevier-proxy-reaxys-parameter-0, elsevier-proxy-reaxys-higestclinicalphase-4, elsevier-proxy-reaxys-chemicalentityname-2, elsevier-proxy-reaxys-proteinassembly-3, elsevier-proxy-reaxys-administrationroute-1, elsevier-proxy-reaxys-compoundtype-1, elsevier-proxy-reaxys-bioassay-1, elsevier-proxy-reaxys-putativeactiontarget-4, elsevier-proxy-reaxys-tissuetype-2, elsevier-proxy-reaxys-reactiontype-1, elsevier-proxy-reaxys-agentconfiguration-2, elsevier-proxy-reaxys-biologicalmaterial-2, elsevier-proxy-reaxys-document-1, elsevier-proxy-reaxys-parameter-1, elsevier-proxy-reaxys-proteinsubunit-0, elsevier-proxy-reaxys-bioassay-0, elsevier-proxy-reaxys-protein-3, elsevier-proxy-reaxys-chemicalentityname-3, elsevier-proxy-reaxys-bioactivity-1, elsevier-proxy-reaxys-administrationroute-0, elsevier-proxy-reaxys-proteinassembly-2, elsevier-proxy-reaxys-compoundtype-2, elsevier-proxy-reaxys-measuresettings-0, elsevier-proxy-reaxys-putativeactiontarget-3, elsevier-proxy-reaxys-tissuetype-1, elsevier-proxy-reaxys-higestclinicalphase-3, elsevier-proxy-reaxys-agentconfiguration-1, elsevier-proxy-reaxys-reactiontype-2, elsevier-proxy-reaxys-biologicalmaterial-1, elsevier-proxy-reaxys-chemicalentityname-4, elsevier-proxy-reaxys-protein-2, elsevier-proxy-reaxys-parameter-2, elsevier-proxy-reaxys-celltype-0, elsevier-proxy-reaxys-bioactivity-2, elsevier-proxy-fda-spl-document-0, elsevier-proxy-reaxys-metabolicreaction-1, elsevier-proxy-reaxys-administrationroute-3, elsevier-proxy-reaxys-proteinsubunit-1, elsevier-proxy-reaxys-organism-3, elsevier-proxy-reaxys-tissuetype-4, elsevier-proxy-reaxys-formulainformation-0, elsevier-proxy-reaxys-chemicalentity-1, elsevier-proxy-reaxys-bioactivity-3, elsevier-proxy-reaxys-agentconfiguration-4, elsevier-proxy-reaxys-biologicalmaterial-4, elsevier-proxy-reaxys-proteinnature-0, elsevier-proxy-reaxys-agentrole-0, elsevier-proxy-reaxys-higestclinicalphase-2, elsevier-proxy-reaxys-document-4, elsevier-proxy-reaxys-parameter-3, elsevier-proxy-reaxys-protein-1, elsevier-proxy-reaxys-metabolicreaction-0, elsevier-proxy-reaxys-proteinassembly-4, elsevier-proxy-reaxys-administrationroute-2, elsevier-proxy-reaxys-compoundtype-0, elsevier-proxy-reaxys-proteinsubunit-2, elsevier-proxy-reaxys-organism-4, elsevier-proxy-reaxys-tissuetype-3, elsevier-proxy-reaxys-formulainformation-1, elsevier-proxy-reaxys-bioactivity-4, elsevier-proxy-reaxys-higestclinicalphase-1, elsevier-proxy-reaxys-agentconfiguration-3, elsevier-proxy-reaxys-proteinnature-1, elsevier-proxy-reaxys-reactiontype-0, elsevier-proxy-reaxys-biologicalmaterial-3, elsevier-proxy-reaxys-document-3, elsevier-proxy-reaxys-chemicalentity-0, elsevier-proxy-reaxys-protein-0, elsevier-proxy-reaxys-parameter-4, elsevier-proxy-reaxys-celltype-2, elsevier-proxy-reaxys-organism-1, elsevier-proxy-reaxys-metabolicreaction-3, elsevier-proxy-reaxys-measuresettings-3, elsevier-proxy-reaxys-proteinsubunit-3, elsevier-proxy-reaxys-formulainformation-2, elsevier-proxy-reaxys-putativeactiontarget-0, elsevier-proxy-reaxys-agentrole-2, elsevier-proxy-reaxys-higestclinicalphase-0, elsevier-proxy-reaxys-chemicalentity-3, elsevier-proxy-reaxys-proteinnature-2, elsevier-proxy-reaxys-celltype-1, elsevier-proxy-reaxys-metabolicreaction-2, elsevier-proxy-reaxys-measuresettings-4, elsevier-proxy-reaxys-administrationroute-4, elsevier-proxy-reaxys-proteinsubunit-4, elsevier-proxy-reaxys-organism-2, elsevier-proxy-reaxys-bioassay-4, elsevier-proxy-reaxys-formulainformation-3, elsevier-proxy-reaxys-chemicalentity-2, elsevier-proxy-reaxys-agentrole-1, elsevier-proxy-reaxys-proteinnature-3, elsevier-proxy-reaxys-document-0, elsevier-proxy-reaxys-chemicalentityname-0, elsevier-proxy-reaxys-measuresettings-1, elsevier-proxy-reaxys-proteinassembly-1, elsevier-proxy-reaxys-compoundtype-3, elsevier-proxy-reaxys-bioassay-3, elsevier-proxy-reaxys-putativeactiontarget-2, elsevier-proxy-reaxys-formulainformation-4, elsevier-proxy-reaxys-tissuetype-0, elsevier-proxy-reaxys-reactiontype-3, elsevier-proxy-reaxys-agentconfiguration-0, elsevier-proxy-reaxys-biologicalmaterial-0, elsevier-proxy-reaxys-proteinnature-4, elsevier-proxy-reaxys-agentrole-4, elsevier-proxy-reaxys-celltype-4, elsevier-proxy-reaxys-celltype-3, elsevier-proxy-reaxys-organism-0, elsevier-proxy-reaxys-chemicalentityname-1, elsevier-proxy-reaxys-metabolicreaction-4, elsevier-proxy-reaxys-measuresettings-2, elsevier-proxy-reaxys-proteinassembly-0, elsevier-proxy-reaxys-bioassay-2, elsevier-proxy-reaxys-compoundtype-4, elsevier-proxy-reaxys-putativeactiontarget-1, elsevier-proxy-reaxys-reactiontype-4, elsevier-proxy-reaxys-chemicalentity-4, elsevier-proxy-reaxys-agentrole-3]
15:06:35.653 [default-akka.kafka.default-dispatcher-19] INFO o.a.k.c.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=elsevier-triple-store-builder-26] Resetting offset for partition elsevier-proxy-reaxys-putativeactiontarget-3 to offset 0.
15:06:35.653 [default-akka.kafka.default-dispatcher-19] INFO o.a.k.c.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=elsevier-triple-store-builder-26] Resetting offset for partition elsevier-proxy-reaxys-compoundtype-0 to offset 0.
15:06:35.653 [default-akka.kafka.default-dispatcher-19] INFO o.a.k.c.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=elsevier-triple-store-builder-26] Resetting offset for partition elsevier-proxy-reaxys-higestclinicalphase-2 to offset 0.
15:06:35.653 [default-akka.kafka.default-dispatcher-19] INFO o.a.k.c.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=elsevier-triple-store-builder-26] Resetting offset for partition elsevier-proxy-reaxys-compoundtype-3 to offset 0.
15:06:35.653 [default-akka.kafka.default-dispatcher-19] INFO o.a.k.c.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=elsevier-triple-store-builder-26] Resetting offset for partition elsevier-proxy-reaxys-proteinnature-2 to offset 0.
15:06:35.657 [default-akka.kafka.default-dispatcher-19] INFO o.a.k.c.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=elsevier-triple-store-builder-26] Resetting offset for partition elsevier-proxy-reaxys-higestclinicalphase-4 to offset 0.
15:06:35.657 [default-akka.kafka.default-dispatcher-19] INFO o.a.k.c.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=elsevier-triple-store-builder-26] Resetting offset for partition elsevier-proxy-reaxys-proteinnature-1 to offset 0.
15:06:35.657 [default-akka.kafka.default-dispatcher-19] INFO o.a.k.c.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=elsevier-triple-store-builder-26] Resetting offset for partition elsevier-proxy-fda-spl-document-0 to offset 0.
15:06:35.657 [default-akka.kafka.default-dispatcher-19] INFO o.a.k.c.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=elsevier-triple-store-builder-26] Resetting offset for partition elsevier-proxy-reaxys-compoundtype-2 to offset 0.
15:06:35.657 [default-akka.kafka.default-dispatcher-19] INFO o.a.k.c.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=elsevier-triple-store-builder-26] Resetting offset for partition elsevier-proxy-reaxys-proteinnature-4 to offset 0.
15:06:35.657 [default-akka.kafka.default-dispatcher-19] INFO o.a.k.c.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=elsevier-triple-store-builder-26] Resetting offset for partition elsevier-proxy-reaxys-putativeactiontarget-4 to offset 0.
15:06:35.657 [default-akka.kafka.default-dispatcher-19] INFO o.a.k.c.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=elsevier-triple-store-builder-26] Resetting offset for partition elsevier-proxy-reaxys-proteinnature-0 to offset 0.
15:06:35.657 [default-akka.kafka.default-dispatcher-19] INFO o.a.k.c.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=elsevier-triple-store-builder-26] Resetting offset for partition elsevier-proxy-reaxys-higestclinicalphase-3 to offset 0.
15:06:35.657 [default-akka.kafka.default-dispatcher-19] INFO o.a.k.c.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=elsevier-triple-store-builder-26] Resetting offset for partition elsevier-proxy-reaxys-compoundtype-1 to offset 0.
15:06:35.657 [default-akka.kafka.default-dispatcher-19] INFO o.a.k.c.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=elsevier-triple-store-builder-26] Resetting offset for partition elsevier-proxy-reaxys-proteinnature-3 to offset 0.
15:06:35.657 [default-akka.kafka.default-dispatcher-19] INFO o.a.k.c.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=elsevier-triple-store-builder-26] Resetting offset for partition elsevier-proxy-reaxys-compoundtype-4 to offset 0.
15:06:35.657 [default-akka.kafka.default-dispatcher-19] INFO o.a.k.c.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=elsevier-triple-store-builder-26] Resetting offset for partition elsevier-proxy-reaxys-agentrole-3 to offset 0.
15:11:26.880 [default-akka.kafka.default-dispatcher-17] INFO o.a.k.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=elsevier-triple-store-builder-26] Node 0 was unable to process the fetch request with (sessionId=2019035881, epoch=29): FETCH_SESSION_ID_NOT_FOUND.
15:11:36.913 [default-akka.kafka.default-dispatcher-8] INFO o.a.k.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=elsevier-triple-store-builder-26] Node 2 was unable to process the fetch request with (sessionId=295384863, epoch=30): FETCH_SESSION_ID_NOT_FOUND.
15:11:36.913 [default-akka.kafka.default-dispatcher-8] INFO o.a.k.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=elsevier-triple-store-builder-26] Node 1 was unable to process the fetch request with (sessionId=1568797267, epoch=30): FETCH_SESSION_ID_NOT_FOUND.
15:16:08.090 [default-akka.kafka.default-dispatcher-17] INFO o.a.k.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=elsevier-triple-store-builder-26] Node 0 was unable to process the fetch request with (sessionId=424717464, epoch=28): FETCH_SESSION_ID_NOT_FOUND.
15:16:38.122 [default-akka.kafka.default-dispatcher-9] INFO o.a.k.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=elsevier-triple-store-builder-26] Node 2 was unable to process the fetch request with (sessionId=1878455433, epoch=30): FETCH_SESSION_ID_NOT_FOUND.
15:16:38.123 [default-akka.kafka.default-dispatcher-9] INFO o.a.k.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=elsevier-triple-store-builder-26] Node 1 was unable to process the fetch request with (sessionId=1382963023, epoch=30): FETCH_SESSION_ID_NOT_FOUND.
15:20:38.890 [default-akka.kafka.default-dispatcher-13] INFO o.a.k.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=elsevier-triple-store-builder-26] Node 0 was unable to process the fetch request with (sessionId=861824997, epoch=27): FETCH_SESSION_ID_NOT_FOUND.
15:21:39.033 [default-akka.kafka.default-dispatcher-11] INFO o.a.k.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=elsevier-triple-store-builder-26] Node 2 was unable to process the fetch request with (sessionId=2058171454, epoch=30): FETCH_SESSION_ID_NOT_FOUND.
15:21:39.034 [default-akka.kafka.default-dispatcher-11] INFO o.a.k.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=elsevier-triple-store-builder-26] Node 1 was unable to process the fetch request with (sessionId=750170814, epoch=30): FETCH_SESSION_ID_NOT_FOUND.
我不知道问题来自哪里,甚至不知道从哪里开始。完全披露我正在使用Akka-stream Kafka(alpakka)版本1.0.5,以及客户端2.1.1,而我的经纪人的版本是2.3.0。
请注意,我的主题每个都有30个分区。我要同时订阅多个主题。