Spring Cloud DataFlow Kubernetes Server无法使用有效的docker URI

时间:2018-03-12 15:38:30

标签: spring-cloud spring-cloud-dataflow

我正在将Spring Cloud DataFlow Kubernetes Server部署从1.2.2.RELEASE升级到1.3.1.RELEASE,并使用minikube对其进行测试。

在修复无效的docker URI(docker:to docker://)并成功查看在shell中注册的应用程序后,对kubernetes服务器的部署最终失败。使用简单的流:

mysource | myprocessor | rabbitsink

Spring Boot sink app 根据pod的日志成功部署并启动。这是该应用程序的成功注册:

app register --name rabbitsink     --type sink       --uri docker://192.168.99.100:80/myrepo/rabbitsink:current
Successfully registered application 'sink:rabbitsink'

尽管如此,数据流shell的后续输出是:

Command failed org.springframework.cloud.dataflow.rest.client.DataFlowClientException: Invalid docker resource URI: docker:192.168.99.100:80/myrepo/rabbitsink:current

并且该流的其他应用都没有启动。

这里是日志中的堆栈跟踪:

2018-03-12 13:17:16.470  INFO 1 --- [           main] s.b.c.e.t.TomcatEmbeddedServletContainer : Tomcat started on port(s): 9393 (http)
2018-03-12 13:17:16.516  INFO 1 --- [           main] o.s.c.d.s.k.KubernetesDataFlowServer     : Started KubernetesDataFlowServer in 54.261 seconds (JVM running for 59.213)
2018-03-12 13:17:49.724  INFO 1 --- [nio-9393-exec-1] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring FrameworkServlet 'dispatcherServlet'
2018-03-12 13:17:53.604  INFO 1 --- [nio-9393-exec-1] .s.c.d.s.s.i.AppDeploymentRequestCreator : Creating resource with [docker://192.168.99.100:80/myrepo/rabbitsink:current] for application [rabbitsink]
2018-03-12 13:17:53.618  INFO 1 --- [nio-9393-exec-1] .s.c.d.s.s.i.AppDeploymentRequestCreator : Creating resource with [docker://192.168.99.100:80/myrepo/myprocessor:current] for application [myprocessor]
2018-03-12 13:17:53.624  INFO 1 --- [nio-9393-exec-1] .s.c.d.s.s.i.AppDeploymentRequestCreator : Creating resource with [docker://192.168.99.100:80/myrepo/mysource:current] for application [mysource]
2018-03-12 13:17:53.629  INFO 1 --- [nio-9393-exec-1] o.s.c.d.s.s.AppDeployerStreamDeployer    : Deploying application named [rabbitsink] as part of stream named [mystream] with resource URI [docker:192.168.99.100:80/myrepo/rabbitsink:current]
2018-03-12 13:17:54.833  INFO 1 --- [nio-9393-exec-1] o.s.c.d.s.k.DefaultContainerFactory      : Using Docker image: 192.168.99.100:80/myrepo/rabbitsink:current
2018-03-12 13:17:54.837  INFO 1 --- [nio-9393-exec-1] o.s.c.d.s.k.DefaultContainerFactory      : Using Docker entry point style: exec
2018-03-12 13:17:58.157  INFO 1 --- [nio-9393-exec-1] o.s.c.d.s.s.AppDeployerStreamDeployer    : Deploying application named [myprocessor] as part of stream named [mystream] with resource URI [docker:192.168.99.100:80/myrepo/myprocessor:current]
2018-03-12 13:17:58.935  INFO 1 --- [nio-9393-exec-1] o.s.c.d.s.k.DefaultContainerFactory      : Using Docker image: 192.168.99.100:80/myrepo/myprocessor:current
2018-03-12 13:17:58.935  INFO 1 --- [nio-9393-exec-1] o.s.c.d.s.k.DefaultContainerFactory      : Using Docker entry point style: exec
2018-03-12 13:18:04.523  INFO 1 --- [nio-9393-exec-1] o.s.c.d.s.s.AppDeployerStreamDeployer    : Deploying application named [mysource] as part of stream named [mystream] with resource URI [docker:192.168.99.100:80/myrepo/mysource:current]
2018-03-12 13:18:05.380  INFO 1 --- [nio-9393-exec-1] o.s.c.d.s.k.DefaultContainerFactory      : Using Docker image: 192.168.99.100:80/myrepo/mysource:current
2018-03-12 13:18:05.419  INFO 1 --- [nio-9393-exec-1] o.s.c.d.s.k.DefaultContainerFactory      : Using Docker entry point style: exec
2018-03-12 13:18:17.497 ERROR 1 --- [nio-9393-exec-1] o.s.c.d.s.c.RestControllerAdvice         : Caught exception while handling a request

java.lang.IllegalArgumentException: Invalid docker resource URI: docker:192.168.99.100:80/myrepo/rabbitsink:current
    at org.springframework.util.Assert.isTrue(Assert.java:92) ~[spring-core-4.3.13.RELEASE.jar!/:4.3.13.RELEASE]
    at org.springframework.cloud.dataflow.registry.support.ResourceUtils.formatDockerResource(ResourceUtils.java:173) ~[spring-cloud-dataflow-registry-1.3.1.RELEASE.jar!/:1.3.1.RELEASE]
    at org.springframework.cloud.dataflow.registry.support.ResourceUtils.getResourceVersion(ResourceUtils.java:147) ~[spring-cloud-dataflow-registry-1.3.1.RELEASE.jar!/:1.3.1.RELEASE]
    at org.springframework.cloud.dataflow.server.stream.AppDeployerStreamDeployer.deployStream(AppDeployerStreamDeployer.java:141) ~[spring-cloud-dataflow-server-core-1.3.1.RELEASE.jar!/:1.3.1.RELEASE]
    at org.springframework.cloud.dataflow.server.service.impl.AppDeployerStreamService.doDeployStream(AppDeployerStreamService.java:88) ~[spring-cloud-dataflow-server-core-1.3.1.RELEASE.jar!/:1.3.1.RELEASE]
    at org.springframework.cloud.dataflow.server.service.impl.AbstractStreamService.deployStream(AbstractStreamService.java:85) ~[spring-cloud-dataflow-server-core-1.3.1.RELEASE.jar!/:1.3.1.RELEASE]
    at org.springframework.cloud.dataflow.server.controller.StreamDeploymentController.deploy(StreamDeploymentController.java:153) ~[spring-cloud-dataflow-server-core-1.3.1.RELEASE.jar!/:1.3.1.RELEASE]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_111-internal]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_111-internal]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_111-internal]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_111-internal]
    at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205) ~[spring-web-4.3.13.RELEASE.jar!/:4.3.13.RELEASE]
    at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133) ~[spring-web-4.3.13.RELEASE.jar!/:4.3.13.RELEASE]
    at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97) ~[spring-webmvc-4.3.13.RELEASE.jar!/:4.3.13.RELEASE]
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827) ~[spring-webmvc-4.3.13.RELEASE.jar!/:4.3.13.RELEASE]
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738) ~[spring-webmvc-4.3.13.RELEASE.jar!/:4.3.13.RELEASE]
    at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85) ~[spring-webmvc-4.3.13.RELEASE.jar!/:4.3.13.RELEASE]
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967) ~[spring-webmvc-4.3.13.RELEASE.jar!/:4.3.13.RELEASE]
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901) ~[spring-webmvc-4.3.13.RELEASE.jar!/:4.3.13.RELEASE]
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970) [spring-webmvc-4.3.13.RELEASE.jar!/:4.3.13.RELEASE]
    at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:872) [spring-webmvc-4.3.13.RELEASE.jar!/:4.3.13.RELEASE]
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:661) [tomcat-embed-core-8.5.23.jar!/:8.5.23]
    at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846) [spring-webmvc-4.3.13.RELEASE.jar!/:4.3.13.RELEASE]
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:742) [tomcat-embed-core-8.5.23.jar!/:8.5.23]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) [tomcat-embed-core-8.5.23.jar!/:8.5.23]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.23.jar!/:8.5.23]
    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) [tomcat-embed-websocket-8.5.23.jar!/:8.5.23]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.23.jar!/:8.5.23]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.23.jar!/:8.5.23]
    at org.springframework.boot.web.filter.ApplicationContextHeaderFilter.doFilterInternal(ApplicationContextHeaderFilter.java:55) [spring-boot-1.5.9.RELEASE.jar!/:1.5.9.RELEASE]
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.13.RELEASE.jar!/:4.3.13.RELEASE]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.23.jar!/:8.5.23]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.23.jar!/:8.5.23]
    at org.springframework.boot.actuate.trace.WebRequestTraceFilter.doFilterInternal(WebRequestTraceFilter.java:110) [spring-boot-actuator-1.5.9.RELEASE.jar!/:1.5.9.RELEASE]
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.13.RELEASE.jar!/:4.3.13.RELEASE]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.23.jar!/:8.5.23]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.23.jar!/:8.5.23]
    at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:208) [spring-security-web-4.2.3.RELEASE.jar!/:4.2.3.RELEASE]
    at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:177) [spring-security-web-4.2.3.RELEASE.jar!/:4.2.3.RELEASE]
    at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:347) [spring-web-4.3.13.RELEASE.jar!/:4.3.13.RELEASE]
    at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:263) [spring-web-4.3.13.RELEASE.jar!/:4.3.13.RELEASE]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.23.jar!/:8.5.23]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.23.jar!/:8.5.23]
    at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99) [spring-web-4.3.13.RELEASE.jar!/:4.3.13.RELEASE]
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.13.RELEASE.jar!/:4.3.13.RELEASE]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.23.jar!/:8.5.23]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.23.jar!/:8.5.23]
    at org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:108) [spring-web-4.3.13.RELEASE.jar!/:4.3.13.RELEASE]
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.13.RELEASE.jar!/:4.3.13.RELEASE]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.23.jar!/:8.5.23]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.23.jar!/:8.5.23]
    at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:81) [spring-web-4.3.13.RELEASE.jar!/:4.3.13.RELEASE]
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.13.RELEASE.jar!/:4.3.13.RELEASE]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.23.jar!/:8.5.23]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.23.jar!/:8.5.23]
    at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:197) [spring-web-4.3.13.RELEASE.jar!/:4.3.13.RELEASE]
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.13.RELEASE.jar!/:4.3.13.RELEASE]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.23.jar!/:8.5.23]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.23.jar!/:8.5.23]
    at org.springframework.boot.actuate.autoconfigure.MetricsFilter.doFilterInternal(MetricsFilter.java:106) [spring-boot-actuator-1.5.9.RELEASE.jar!/:1.5.9.RELEASE]
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.13.RELEASE.jar!/:4.3.13.RELEASE]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.23.jar!/:8.5.23]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.23.jar!/:8.5.23]
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199) [tomcat-embed-core-8.5.23.jar!/:8.5.23]
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) [tomcat-embed-core-8.5.23.jar!/:8.5.23]
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:478) [tomcat-embed-core-8.5.23.jar!/:8.5.23]
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140) [tomcat-embed-core-8.5.23.jar!/:8.5.23]
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81) [tomcat-embed-core-8.5.23.jar!/:8.5.23]
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87) [tomcat-embed-core-8.5.23.jar!/:8.5.23]
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342) [tomcat-embed-core-8.5.23.jar!/:8.5.23]
    at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:803) [tomcat-embed-core-8.5.23.jar!/:8.5.23]
    at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) [tomcat-embed-core-8.5.23.jar!/:8.5.23]
    at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868) [tomcat-embed-core-8.5.23.jar!/:8.5.23]
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1459) [tomcat-embed-core-8.5.23.jar!/:8.5.23]
    at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) [tomcat-embed-core-8.5.23.jar!/:8.5.23]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_111-internal]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_111-internal]
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-embed-core-8.5.23.jar!/:8.5.23]
    at java.lang.Thread.run(Thread.java:745) [na:1.8.0_111-internal]

3 个答案:

答案 0 :(得分:0)

我们最近在spring-cloud/spring-cloud-dataflow#2065中讨论了类似的问题。

正如comment中所强调的那样,Docker资源URI应该是从1.3版本开始的某种格式。

在您的情况下,Docker URI应该是:

  

dataflow:> app register --name rabbitsink --type sink --uri docker:myrepo / rabbitsink:current

如果未遵循上述格式,则Shell将通过验证消息快速失败。流不会被部署;也不应该为应用程序显示任何pod。这是我的minikube环境中的验证消息。

  

命令失败org.springframework.cloud.dataflow.rest.client.DataFlowClientException:无效的docker资源URI:docker:192.168.99.100:80 / myrepo / rabbitsink:current

我建议重新开始。您还可以使用helm-chartDocs for Minikube is in PR)来简化配置体验。

答案 1 :(得分:0)

这是我需要解决的错误。测试

    @Test
public void testDockerResourceProcessingSO() {
    DockerResource dockerResource = new DockerResource("192.168.99.100:80/myrepo/rabbitsink:current");
    assertThat(ResourceUtils.getResourceWithoutVersion(dockerResource)).isEqualTo("docker:192.168.99.100:80/myrepo/rabbitsink");
    assertThat(ResourceUtils.getResourceVersion(dockerResource)).isEqualTo("current");
}

失败。由于这行代码

            String dockerResourceUri = dockerResource.getURI().toString();
        Assert.isTrue(StringUtils.countOccurrencesOf(dockerResourceUri, ":") == 2,
                "Invalid docker resource URI: " + dockerResourceUri)

和dockerResourceUri是docker:192.168.99.100:80 / myrepo / rabbitsink:current

如果使用了船长,还会有另一个问题,因为版本需要遵循' semver'格式和'当前'不会有效。我们可以更好地验证docker格式并尽快修复。

我们解决了这个问题后,如果你使用了船长,那么船长就不会喜欢“当前的'作为版本名称,它必须是真实的版本号。尚不确定我们是否可以采取任何措施来适应“开发”工作流程'当船长的整个要点是管理版本更新时没有版本号。也许我们需要允许“当前'”,“最新'' SNAPSHOT'作为允许的版本,并始终在请求更新时重新部署这些应用程序。

答案 2 :(得分:0)

这已在1.4.0.RELEASE版本的Spring Cloud Dataflow中修复,并且与本地数据流服务器的1.4.0.RELEASE版本,kubernetes数据流服务器和数据流shell一起使用时效果很好。它适用于Spring Cloud Edgware.SR2和Edgware.SR3。