kafka.admin.AdminClient.listAllConsumerGroups（）有时会失败

时间：2017-06-30 15:48:34

标签： java apache-kafka

我一直在尝试使用AdminClient类获取所有使用者组，但有时我会收到运行时异常。我正在使用带有Angular前端的spring启动应用程序并发出一个http请求来调用此方法：

@RequestMapping("groupIds")
    public List<String> getGroupIds() {

        /*get all of the consumer groups and convert the scala map to a java map*/
        Map<Node, scala.collection.immutable.List<GroupOverview>> consumerGroups = scala.collection.JavaConverters
                .mapAsJavaMapConverter(adminClient.listAllConsumerGroups()).asJava();

        /*Remove the GroupOverviews from the Map*/
        List<GroupOverview> consumerGroupList = getConsumerGroupOverviewList(consumerGroups);

        /*Remove the groupIds from the GroupOverviews*/
        return getGroupIdsFromGroupOverview(consumerGroupList);
    }

大部分时间它工作正常，但有时会失败。我似乎无法弄清楚是什么导致它，但它是通过重新加载页面来解决的。这是堆栈跟踪：

org.springframework.web.util.NestedServletException: Request processing failed;
nested exception is java.lang.RuntimeException: Request METADATA failed on broke
rs List(localhost:9092 (id: -1 rack: null))
        at org.springframework.web.servlet.FrameworkServlet.processRequest(Frame
workServlet.java:982) ~[spring-webmvc-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServl
et.java:861) ~[spring-webmvc-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:687) ~[javax.
servlet-api-3.1.0.jar:3.1.0]
        at org.springframework.web.servlet.FrameworkServlet.service(FrameworkSer
vlet.java:846) ~[spring-webmvc-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) ~[javax.
servlet-api-3.1.0.jar:3.1.0]
        at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:841
) ~[jetty-servlet-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(Servlet
Handler.java:1634) ~[jetty-servlet-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(We
bSocketUpgradeFilter.java:206) ~[websocket-server-9.4.2.v20170220.jar:9.4.2.v201
70220]
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(Servlet
Handler.java:1621) ~[jetty-servlet-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.springframework.boot.web.filter.ApplicationContextHeaderFilter.do
FilterInternal(ApplicationContextHeaderFilter.java:55) ~[spring-boot-1.5.2.RELEA
SE.jar:1.5.2.RELEASE]
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerR
equestFilter.java:107) ~[spring-web-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(Servlet
Handler.java:1621) ~[jetty-servlet-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.springframework.boot.actuate.trace.WebRequestTraceFilter.doFilter
Internal(WebRequestTraceFilter.java:108) ~[spring-boot-actuator-1.5.2.RELEASE.ja
r:1.5.2.RELEASE]
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerR
equestFilter.java:107) ~[spring-web-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(Servlet
Handler.java:1621) ~[jetty-servlet-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.springframework.web.filter.RequestContextFilter.doFilterInternal(
RequestContextFilter.java:99) ~[spring-web-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerR
equestFilter.java:107) ~[spring-web-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(Servlet
Handler.java:1621) ~[jetty-servlet-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.springframework.web.filter.HttpPutFormContentFilter.doFilterInter
nal(HttpPutFormContentFilter.java:105) ~[spring-web-4.3.7.RELEASE.jar:4.3.7.RELE
ASE]
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerR
equestFilter.java:107) ~[spring-web-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(Servlet
Handler.java:1621) ~[jetty-servlet-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInterna
l(HiddenHttpMethodFilter.java:81) ~[spring-web-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerR
equestFilter.java:107) ~[spring-web-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(Servlet
Handler.java:1621) ~[jetty-servlet-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.springframework.web.filter.CharacterEncodingFilter.doFilterIntern
al(CharacterEncodingFilter.java:197) ~[spring-web-4.3.7.RELEASE.jar:4.3.7.RELEAS
E]
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerR
equestFilter.java:107) ~[spring-web-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(Servlet
Handler.java:1621) ~[jetty-servlet-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.springframework.boot.actuate.autoconfigure.MetricsFilter.doFilter
Internal(MetricsFilter.java:106) ~[spring-boot-actuator-1.5.2.RELEASE.jar:1.5.2.
RELEASE]
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerR
equestFilter.java:107) ~[spring-web-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(Servlet
Handler.java:1621) ~[jetty-servlet-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java
:541) ~[jetty-servlet-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.j
ava:143) ~[jetty-server-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.jav
a:548) ~[jetty-security-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper
.java:132) ~[jetty-server-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandl
er.java:190) ~[jetty-server-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandl
er.java:1592) ~[jetty-server-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandl
er.java:188) ~[jetty-server-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandl
er.java:1239) ~[jetty-server-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandle
r.java:168) ~[jetty-server-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:
481) ~[jetty-servlet-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandle
r.java:1561) ~[jetty-server-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandle
r.java:166) ~[jetty-server-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandle
r.java:1141) ~[jetty-server-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.j
ava:141) ~[jetty-server-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper
.java:132) ~[jetty-server-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.eclipse.jetty.server.Server.handle(Server.java:564) ~[jetty-serve
r-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320) ~[j
etty-server-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.jav
a:251) [jetty-server-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(Abstra
ctConnection.java:279) [jetty-io-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:110) [je
tty-io-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
[jetty-io-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.eclipse.jetty.util.thread.Invocable.invokePreferred(Invocable.jav
a:122) [jetty-util-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.eclipse.jetty.util.thread.strategy.ExecutingExecutionStrategy.inv
oke(ExecutingExecutionStrategy.java:58) [jetty-util-9.4.2.v20170220.jar:9.4.2.v2
0170220]
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceC
onsume(ExecuteProduceConsume.java:201) [jetty-util-9.4.2.v20170220.jar:9.4.2.v20
170220]
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(Exec
uteProduceConsume.java:133) [jetty-util-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPoo
l.java:672) [jetty-util-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool
.java:590) [jetty-util-9.4.2.v20170220.jar:9.4.2.v20170220]
        at java.lang.Thread.run(Thread.java:748) [na:1.8.0_131]
Caused by: java.lang.RuntimeException: Request METADATA failed on brokers List(l
ocalhost:9092 (id: -1 rack: null))
        at kafka.admin.AdminClient.sendAnyNode(AdminClient.scala:66) ~[kafka_2.1
2-0.10.2.1.jar:na]
        at kafka.admin.AdminClient.findAllBrokers(AdminClient.scala:90) ~[kafka_
2.12-0.10.2.1.jar:na]
        at kafka.admin.AdminClient.listAllGroups(AdminClient.scala:98) ~[kafka_2
.12-0.10.2.1.jar:na]
        at kafka.admin.AdminClient.listAllConsumerGroups(AdminClient.scala:112)
~[kafka_2.12-0.10.2.1.jar:na]
        at consumer.KafkaController.getGroupIds(KafkaController.java:154) ~[main
/:na]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.
0_131]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
java:62) ~[na:1.8.0_131]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
sorImpl.java:43) ~[na:1.8.0_131]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_131]
        at org.springframework.web.method.support.InvocableHandlerMethod.doInvok
e(InvocableHandlerMethod.java:205) ~[spring-web-4.3.7.RELEASE.jar:4.3.7.RELEASE]

        at org.springframework.web.method.support.InvocableHandlerMethod.invokeF
orRequest(InvocableHandlerMethod.java:133) ~[spring-web-4.3.7.RELEASE.jar:4.3.7.
RELEASE]
        at org.springframework.web.servlet.mvc.method.annotation.ServletInvocabl
eHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:116) ~[spring-
webmvc-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at org.springframework.web.servlet.mvc.method.annotation.RequestMappingH
andlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827) ~[sprin
g-webmvc-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at org.springframework.web.servlet.mvc.method.annotation.RequestMappingH
andlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738) ~[spring-web
mvc-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapt
er.handle(AbstractHandlerMethodAdapter.java:85) ~[spring-webmvc-4.3.7.RELEASE.ja
r:4.3.7.RELEASE]
        at org.springframework.web.servlet.DispatcherServlet.doDispatch(Dispatch
erServlet.java:963) ~[spring-webmvc-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at org.springframework.web.servlet.DispatcherServlet.doService(Dispatche
rServlet.java:897) ~[spring-webmvc-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at org.springframework.web.servlet.FrameworkServlet.processRequest(Frame
workServlet.java:970) ~[spring-webmvc-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        ... 57 common frames omitted

2 个答案:

答案 0 :(得分：1)

这似乎是卡夫卡的一个错误。

我曾经提出同样的问题当使用以下命令获取kafka偏移时：

./kafka-consumer-groups.sh --list --new-consumer --bootstrap-server kafka-broker-server:21007 --command-config ../config/consumer.properties

大多数时候，它都有效，但偶尔会失败。

不知道发生了什么。

答案 1 :(得分：1)

我不知道导致问题的是什么，但每次在调用方法之前创建AdminClient似乎已经解决了我的问题。从那以后我从来没有犯过这个错误。因此，如果您在每次调用

之前创建adminClient 
Map<Node, scala.collection.immutable.List<GroupOverview>> consumerGroups = scala.collection.JavaConverters
                .mapAsJavaMapConverter(adminClient.listAllConsumerGroups()).asJava();

你永远不应该看到这个问题。

相关问题
最新问题