我正在尝试使用Spring Integration和Docker上的Kafka实现以下场景:
轮询工作正常,我的应用程序设置似乎是正确的,因为应用程序启动时没有任何连接异常。
正如您在输出中看到的那样,KafkaProducerMessageHandler接收消息,但挂起“尝试向节点-1发送元数据请求”超过一分钟,然后抛出TimeoutException。
以下是链接所需的输出和代码:
这是我的应用程序设置:
KafkaConfig基于https://github.com/spring-projects/spring-integration-samples的Kafka示例和https://github.com/wurstmeister/kafka-docker的Kafka Docker容器。
感谢您的任何想法和建议!!!!
答案 0 :(得分:1)
我们在您的日志中看到类似的内容:
227 --- [kafka-producer-network-thread | producer-1] org.apache.kafka.clients.NetworkClient : Sending metadata request ClientRequest(expectResponse=true, payload=null, request=RequestSend(header={api_key=3,api_version=0,correlation_id=23,client_id=producer-1}, body={topics=[test.topic]})) to node -1
2016-01-26 15:22:24.986 DEBUG 4227 --- [kafka-producer-network-thread | producer-1] org.apache.kafka.clients.NetworkClient : Trying to send metadata request to node -1
看起来你真的应该查看Broker日志。
首先它可能是:
kafka.common.InvalidTopicException: topic name TOPIC= is illegal, contains a character other than ASCII alphanumerics, '.', '_' and '-'
https://issues.apache.org/jira/browse/KAFKA-1884
因此,主题名称中的.
是非法的。您可以将不兼容的Kafka版本与其客户端一起使用。 Spring Integration Kafka 1.3基于Kafka-0.8.2.2。
从另一方面来看,可能存在一些网络问题:Kafka 0.8.2.2 - Unable to publish messages
答案 1 :(得分:1)
我是个傻瓜!
我的码头组成是:
kafka:
image: wurstmeister/kafka
ports:
- "9092:9092"
links:
- zookeeper:zk
environment:
KAFKA_ADVERTISED_HOST_NAME: 192.168.59.103
volumes:
- /var/run/docker.sock:/var/run/docker.sock
其中192.168.59.103是我的IP。我不得不把它改成localhost,现在看起来很好。
17:25:26.797 DEBUG 25436 --- [ask-scheduler-9] o.s.integration.channel.DirectChannel : preSend on channel 'kafkaChannel', message: GenericMessage [payload={"login":"markuslamm","id":1075906,"avatar_url":"https://avatars.githubusercontent.com/u/1075906?v=3","gravatar_id":"","url":"https://api.github.com/users/markuslamm","html_url":"https://github.com/markuslamm","followers_url":"https://api.github.com/users/markuslamm/followers","following_url":"https://api.github.com/users/markuslamm/following{/other_user}","gists_url":"https://api.github.com/users/markuslamm/gists{/gist_id}","starred_url":"https://api.github.com/users/markuslamm/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/markuslamm/subscriptions","organizations_url":"https://api.github.com/users/markuslamm/orgs","repos_url":"https://api.github.com/users/markuslamm/repos","events_url":"https://api.github.com/users/markuslamm/events{/privacy}","received_events_url":"https://api.github.com/users/markuslamm/received_events","type":"User","site_admin":false,"name":"Markus Lamm","company":null,"blog":null,"location":null,"email":null,"hireable":null,"bio":null,"public_repos":21,"public_gists":0,"followers":0,"following":0,"created_at":"2011-09-24T07:04:34Z","updated_at":"2016-01-26T15:19:00Z"}, headers={Server=GitHub.com, Cache-Control=public, max-age=60, s-maxage=60, ETag="c06cdef5b902bb019cced50412d4b913", Vary=[Accept, Accept-Encoding], Last-Modified=1453821540000, id=6bffbe99-0e6d-55a5-65a1-208c7be7542d, Content-Length=1135, contentType=application/json;charset=utf-8, http_statusCode=200, Date=1453825526000, timestamp=1453825526797}]
2016-01-26 17:25:26.797 DEBUG 25436 --- [ask-scheduler-9] ssor$ReplyProducingMessageHandlerWrapper : kafkaIntegration.kafkaHandler.serviceActivator.handler received message: GenericMessage [payload={"login":"markuslamm","id":1075906,"avatar_url":"https://avatars.githubusercontent.com/u/1075906?v=3","gravatar_id":"","url":"https://api.github.com/users/markuslamm","html_url":"https://github.com/markuslamm","followers_url":"https://api.github.com/users/markuslamm/followers","following_url":"https://api.github.com/users/markuslamm/following{/other_user}","gists_url":"https://api.github.com/users/markuslamm/gists{/gist_id}","starred_url":"https://api.github.com/users/markuslamm/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/markuslamm/subscriptions","organizations_url":"https://api.github.com/users/markuslamm/orgs","repos_url":"https://api.github.com/users/markuslamm/repos","events_url":"https://api.github.com/users/markuslamm/events{/privacy}","received_events_url":"https://api.github.com/users/markuslamm/received_events","type":"User","site_admin":false,"name":"Markus Lamm","company":null,"blog":null,"location":null,"email":null,"hireable":null,"bio":null,"public_repos":21,"public_gists":0,"followers":0,"following":0,"created_at":"2011-09-24T07:04:34Z","updated_at":"2016-01-26T15:19:00Z"}, headers={Server=GitHub.com, Cache-Control=public, max-age=60, s-maxage=60, ETag="c06cdef5b902bb019cced50412d4b913", Vary=[Accept, Accept-Encoding], Last-Modified=1453821540000, id=6bffbe99-0e6d-55a5-65a1-208c7be7542d, Content-Length=1135, contentType=application/json;charset=utf-8, http_statusCode=200, Date=1453825526000, timestamp=1453825526797}]
2016-01-26 17:25:26.797 DEBUG 25436 --- [ask-scheduler-9] o.s.i.k.o.KafkaProducerMessageHandler : kafkaHandler received message: GenericMessage [payload={"login":"markuslamm","id":1075906,"avatar_url":"https://avatars.githubusercontent.com/u/1075906?v=3","gravatar_id":"","url":"https://api.github.com/users/markuslamm","html_url":"https://github.com/markuslamm","followers_url":"https://api.github.com/users/markuslamm/followers","following_url":"https://api.github.com/users/markuslamm/following{/other_user}","gists_url":"https://api.github.com/users/markuslamm/gists{/gist_id}","starred_url":"https://api.github.com/users/markuslamm/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/markuslamm/subscriptions","organizations_url":"https://api.github.com/users/markuslamm/orgs","repos_url":"https://api.github.com/users/markuslamm/repos","events_url":"https://api.github.com/users/markuslamm/events{/privacy}","received_events_url":"https://api.github.com/users/markuslamm/received_events","type":"User","site_admin":false,"name":"Markus Lamm","company":null,"blog":null,"location":null,"email":null,"hireable":null,"bio":null,"public_repos":21,"public_gists":0,"followers":0,"following":0,"created_at":"2011-09-24T07:04:34Z","updated_at":"2016-01-26T15:19:00Z"}, headers={Server=GitHub.com, Cache-Control=public, max-age=60, s-maxage=60, ETag="c06cdef5b902bb019cced50412d4b913", Vary=[Accept, Accept-Encoding], Last-Modified=1453821540000, id=6bffbe99-0e6d-55a5-65a1-208c7be7542d, Content-Length=1135, contentType=application/json;charset=utf-8, http_statusCode=200, Date=1453825526000, timestamp=1453825526797}]
2016-01-26 17:25:26.798 DEBUG 25436 --- [ask-scheduler-9] ssor$ReplyProducingMessageHandlerWrapper : handler 'kafkaIntegration.kafkaHandler.serviceActivator.handler' produced no reply for request Message: GenericMessage [payload={"login":"markuslamm","id":1075906,"avatar_url":"https://avatars.githubusercontent.com/u/1075906?v=3","gravatar_id":"","url":"https://api.github.com/users/markuslamm","html_url":"https://github.com/markuslamm","followers_url":"https://api.github.com/users/markuslamm/followers","following_url":"https://api.github.com/users/markuslamm/following{/other_user}","gists_url":"https://api.github.com/users/markuslamm/gists{/gist_id}","starred_url":"https://api.github.com/users/markuslamm/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/markuslamm/subscriptions","organizations_url":"https://api.github.com/users/markuslamm/orgs","repos_url":"https://api.github.com/users/markuslamm/repos","events_url":"https://api.github.com/users/markuslamm/events{/privacy}","received_events_url":"https://api.github.com/users/markuslamm/received_events","type":"User","site_admin":false,"name":"Markus Lamm","company":null,"blog":null,"location":null,"email":null,"hireable":null,"bio":null,"public_repos":21,"public_gists":0,"followers":0,"following":0,"created_at":"2011-09-24T07:04:34Z","updated_at":"2016-01-26T15:19:00Z"}, headers={Server=GitHub.com, Cache-Control=public, max-age=60, s-maxage=60, ETag="c06cdef5b902bb019cced50412d4b913", Vary=[Accept, Accept-Encoding], Last-Modified=1453821540000, id=6bffbe99-0e6d-55a5-65a1-208c7be7542d, Content-Length=1135, contentType=application/json;charset=utf-8, http_statusCode=200, Date=1453825526000, timestamp=1453825526797}]
2016-01-26 17:25:26.798 DEBUG 25436 --- [ask-scheduler-9] o.s.integration.channel.DirectChannel : postSend (sent=true) on channel 'kafkaChannel', message: GenericMessage [payload={"login":"markuslamm","id":1075906,"avatar_url":"https://avatars.githubusercontent.com/u/1075906?v=3","gravatar_id":"","url":"https://api.github.com/users/markuslamm","html_url":"https://github.com/markuslamm","followers_url":"https://api.github.com/users/markuslamm/followers","following_url":"https://api.github.com/users/markuslamm/following{/other_user}","gists_url":"https://api.github.com/users/markuslamm/gists{/gist_id}","starred_url":"https://api.github.com/users/markuslamm/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/markuslamm/subscriptions","organizations_url":"https://api.github.com/users/markuslamm/orgs","repos_url":"https://api.github.com/users/markuslamm/repos","events_url":"https://api.github.com/users/markuslamm/events{/privacy}","received_events_url":"https://api.github.com/users/markuslamm/received_events","type":"User","site_admin":false,"name":"Markus Lamm","company":null,"blog":null,"location":null,"email":null,"hireable":null,"bio":null,"public_repos":21,"public_gists":0,"followers":0,"following":0,"created_at":"2011-09-24T07:04:34Z","updated_at":"2016-01-26T15:19:00Z"}, headers={Server=GitHub.com, Cache-Control=public, max-age=60, s-maxage=60, ETag="c06cdef5b902bb019cced50412d4b913", Vary=[Accept, Accept-Encoding], Last-Modified=1453821540000, id=6bffbe99-0e6d-55a5-65a1-208c7be7542d, Content-Length=1135, contentType=application/json;charset=utf-8, http_statusCode=200, Date=1453825526000, timestamp=1453825526797}]
2016-01-26 17:25:26.799 DEBUG 25436 --- [ask-scheduler-9] o.s.integration.channel.DirectChannel : postSend (sent=true) on channel 'httpPollChannel', message: GenericMessage [payload=, headers={id=55aecfc4-ea57-d769-4275-801b39a0416d, timestamp=1453825526420}]