Spring Cloud Task App-组合任务运行程序不会关闭

时间:2019-02-11 09:20:12

标签: spring-cloud-dataflow spring-cloud-task

我在本地环境中的SCDF平台(kubernetes)上创建了一个组合任务。但是我想知道为什么在所有单个任务停止后组合任务不会停止。

spring-cloud-dataflow-server-kubernetes:1.7.2。发布

组合任务运行程序版本:docker:springcloudtask / compositetaskrunner-task:2.1.0.M1

组成的任务定义:

task create --name dbsales-pipeline-st --definition "ftp-source-task-st: ftp-source-task --host=ftp-host --username=ftp-user --port=21 --remote-directory='/trial/' --delete-remote-file=true --password=password --source-exchange-name=xml-transform-exchange-source --source-exchange-routing-key=st --name=ftp-source-task-st && xml-transform-source-st: xml-transform-task --sink-exchange-name=xml-transform-exchange-sink --source-queue-name=xml-transform-source-st --sink-exchange-routing-key=st --name=xml-transform-source-st && <ftp-sink-task-aaa: ftp-sink-task --host=ftp-host --username=ftp-user --port=21 --remote-directory='/trial-output/aaa/' --password=password --source-queue-name=xml-transform-sink-st-aaa --name=ftp-sink-task-aaa || ftp-sink-task-bbb: ftp-sink-task --host=ftp-host --username=ftp-user --port=21 --remote-directory='/trial-output/bbb/' --password=password --source-queue-name=xml-transform-sink-st-bbb --name=ftp-sink-task-bbb >"

Kubernetes日志

NAME                                                     READY     STATUS      RESTARTS   AGE
dbsales-pipeline-st-ez28melww9                           1/1       Running     0          14m
dbsales-pipeline-st-ftp-sink-task-aaa-lwenvxo7ry     0/1       Completed   0          12m
dbsales-pipeline-st-ftp-sink-task-bbb-vmwpnd250d   0/1       Completed   0          12m
dbsales-pipeline-st-ftp-source-task-st-jep9zyzpzv        0/1       Completed   0          14m
dbsales-pipeline-st-xml-transform-source-st-rodzk0voz7   0/1       Completed   0          13m

从组合任务执行中记录日志:

2019-02-11 09:01:34.598  INFO 1 --- [           main] c.c.c.ConfigServicePropertySourceLocator : Fetching config from server at : http://localhost:8888
2019-02-11 09:01:34.828  INFO 1 --- [           main] c.c.c.ConfigServicePropertySourceLocator : Connect Timeout Exception on Url - http://localhost:8888. Will be trying the next url if available
2019-02-11 09:01:34.829  WARN 1 --- [           main] c.c.c.ConfigServicePropertySourceLocator : Could not locate PropertySource: I/O error on GET request for "http://localhost:8888/composedtaskrunner-task/default": Connection refused (Connection refused); nested exception is java.net.ConnectException: Connection refused (Connection refused)
2019-02-11 09:01:34.832  INFO 1 --- [           main] .t.a.c.ComposedtaskrunnerTaskApplication : No active profile set, falling back to default profiles: default
2019-02-11 09:01:36.589  INFO 1 --- [           main] o.s.cloud.context.scope.GenericScope     : BeanFactory id=8b0b06e3-848d-3c1a-9a81-67298b0a58c4
2019-02-11 09:01:36.848  INFO 1 --- [           main] trationDelegate$BeanPostProcessorChecker : Bean 'org.springframework.transaction.annotation.ProxyTransactionManagementConfiguration' of type [org.springframework.transaction.annotation.ProxyTransactionManagementConfiguration$$EnhancerBySpringCGLIB$$c1b98ce] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2019-02-11 09:01:36.943  INFO 1 --- [           main] trationDelegate$BeanPostProcessorChecker : Bean 'org.springframework.cloud.autoconfigure.ConfigurationPropertiesRebinderAutoConfiguration' of type [org.springframework.cloud.autoconfigure.ConfigurationPropertiesRebinderAutoConfiguration$$EnhancerBySpringCGLIB$$28359bcb] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2019-02-11 09:01:36.948  INFO 1 --- [           main] trationDelegate$BeanPostProcessorChecker : Bean 'org.springframework.cloud.task.batch.configuration.TaskBatchAutoConfiguration' of type [org.springframework.cloud.task.batch.configuration.TaskBatchAutoConfiguration$$EnhancerBySpringCGLIB$$918b445a] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2019-02-11 09:01:37.397  INFO 1 --- [           main] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Starting...
2019-02-11 09:01:37.570  INFO 1 --- [           main] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Start completed.
2019-02-11 09:01:37.656  INFO 1 --- [           main] o.s.s.concurrent.ThreadPoolTaskExecutor  : Initializing ExecutorService 'taskExecutor'
2019-02-11 09:01:39.641  INFO 1 --- [           main] o.s.b.c.r.s.JobRepositoryFactoryBean     : No database type set, using meta data indicating: MYSQL
2019-02-11 09:01:39.701  INFO 1 --- [           main] o.s.b.c.l.support.SimpleJobLauncher      : No TaskExecutor has been set, defaulting to synchronous executor.
2019-02-11 09:01:40.381  INFO 1 --- [           main] .t.a.c.ComposedtaskrunnerTaskApplication : Started ComposedtaskrunnerTaskApplication in 8.093 seconds (JVM running for 9.199)
2019-02-11 09:01:40.383  INFO 1 --- [           main] o.s.b.a.b.JobLauncherCommandLineRunner   : Running default command line with: [--spring.datasource.username=root, --spring.datasource.url=jdbc:mysql://10.102.85.203:3306/mysql, --spring.datasource.driverClassName=org.mariadb.jdbc.Driver, --dataflow-server-uri=http://10.99.69.126:80, --spring.cloud.task.name=dbsales-pipeline-st, --graph=dbsales-pipeline-st-ftp-source-task-st && dbsales-pipeline-st-xml-transform-source-st && <dbsales-pipeline-st-ftp-sink-task-aaa || dbsales-pipeline-st-ftp-sink-task-bbb>, --spring.datasource.password=yourpassword, --increment-instance-enabled=true, --split-thread-wait-for-tasks-to-complete-on-shutdown=true, --dataflow-server-username=user, --dataflow-server-password=password, id=1, --spring.cloud.task.executionid=735]
2019-02-11 09:01:40.603  INFO 1 --- [           main] o.s.b.c.l.support.SimpleJobLauncher      : Job: [FlowJob: [name=dbsales-pipeline-st]] launched with the following parameters: [{-spring.cloud.task.executionid=735, -spring.datasource.username=root, -spring.cloud.task.name=dbsales-pipeline-st, -graph=dbsales-pipeline-st-ftp-source-task-st && dbsales-pipeline-st-xml-transform-source-st && <dbsales-pipeline-st-ftp-sink-task-aaa || dbsales-pipeline-st-ftp-sink-task-bbb>, -dataflow-server-password=password, incrementInstanceEnabled=true, -spring.datasource.driverClassName=org.mariadb.jdbc.Driver, -increment-instance-enabled=true, -spring.datasource.url=jdbc:mysql://10.102.85.203:3306/mysql, -dataflow-server-username=user, -split-thread-wait-for-tasks-to-complete-on-shutdown=true, run.id=4, -dataflow-server-uri=http://10.99.69.126:80, id=1, -spring.datasource.password=yourpassword}]
2019-02-11 09:01:40.617  WARN 1 --- [           main] o.s.c.t.b.l.TaskBatchExecutionListener   : This job was executed outside the scope of a task but still used the task listener.
2019-02-11 09:01:40.647  INFO 1 --- [           main] o.s.batch.core.job.SimpleStepHandler     : Executing step: [dbsales-pipeline-st-ftp-source-task-st_0]
2019-02-11 09:03:11.499  INFO 1 --- [           main] .t.a.c.ComposedTaskStepExecutionListener : AfterStep processing for stepExecution dbsales-pipeline-st-ftp-source-task-st_0
2019-02-11 09:03:11.502  INFO 1 --- [           main] .t.a.c.ComposedTaskStepExecutionListener : AfterStep processing complete for stepExecution dbsales-pipeline-st-ftp-source-task-st_0 with taskExecution 736
2019-02-11 09:03:11.526  INFO 1 --- [           main] o.s.batch.core.job.SimpleStepHandler     : Executing step: [dbsales-pipeline-st-xml-transform-source-st_0]
2019-02-11 09:03:22.288  INFO 1 --- [           main] .t.a.c.ComposedTaskStepExecutionListener : AfterStep processing for stepExecution dbsales-pipeline-st-xml-transform-source-st_0
2019-02-11 09:03:22.295  INFO 1 --- [           main] .t.a.c.ComposedTaskStepExecutionListener : AfterStep processing complete for stepExecution dbsales-pipeline-st-xml-transform-source-st_0 with taskExecution 737
2019-02-11 09:03:22.431  INFO 1 --- [ taskExecutor-1] o.s.batch.core.job.SimpleStepHandler     : Executing step: [dbsales-pipeline-st-ftp-sink-task-aaa_0]
2019-02-11 09:03:22.441  INFO 1 --- [ taskExecutor-2] o.s.batch.core.job.SimpleStepHandler     : Executing step: [dbsales-pipeline-st-ftp-sink-task-bbb_0]
2019-02-11 09:04:53.824  INFO 1 --- [ taskExecutor-2] .t.a.c.ComposedTaskStepExecutionListener : AfterStep processing for stepExecution dbsales-pipeline-st-ftp-sink-task-bbb_0
2019-02-11 09:04:53.828  INFO 1 --- [ taskExecutor-2] .t.a.c.ComposedTaskStepExecutionListener : AfterStep processing complete for stepExecution dbsales-pipeline-st-ftp-sink-task-bbb_0 with taskExecution 738
2019-02-11 09:04:53.836  INFO 1 --- [ taskExecutor-1] .t.a.c.ComposedTaskStepExecutionListener : AfterStep processing for stepExecution dbsales-pipeline-st-ftp-sink-task-aaa_0
2019-02-11 09:04:53.840  INFO 1 --- [ taskExecutor-1] .t.a.c.ComposedTaskStepExecutionListener : AfterStep processing complete for stepExecution dbsales-pipeline-st-ftp-sink-task-aaa_0 with taskExecution 739
2019-02-11 09:04:53.864  INFO 1 --- [           main] o.s.b.c.l.support.SimpleJobLauncher      : Job: [FlowJob: [name=dbsales-pipeline-st]] completed with the following parameters: [{-spring.cloud.task.executionid=735, -spring.datasource.username=root, -spring.cloud.task.name=dbsales-pipeline-st, -graph=dbsales-pipeline-st-ftp-source-task-st && dbsales-pipeline-st-xml-transform-source-st && <dbsales-pipeline-st-ftp-sink-task-aaa || dbsales-pipeline-st-ftp-sink-task-bbb>, -dataflow-server-password=password, incrementInstanceEnabled=true, -spring.datasource.driverClassName=org.mariadb.jdbc.Driver, -increment-instance-enabled=true, -spring.datasource.url=jdbc:mysql://10.102.85.203:3306/mysql, -dataflow-server-username=user, -split-thread-wait-for-tasks-to-complete-on-shutdown=true, run.id=4, -dataflow-server-uri=http://10.99.69.126:80, id=1, -spring.datasource.password=yourpassword}] and the following status: [COMPLETED]

我确实尝试了一个简单的组合任务,但它确实停止了。

task create foo --definition "t1: timestamp && t2: timestamp"

是否有任何参数可以迫使合成任务在完成后停止?

1 个答案:

答案 0 :(得分:0)

通常发生的迹象是组合任务运行程序的spring.cloud.task.closecontextEnabled未设置为“ true”时。

您可以在Spring Cloud Task docs中了解有关该属性的更多信息。

最近,在spring-cloud-task-app-starters/composed-task-runner#60spring-cloud/spring-cloud-dataflow#2667上对此进行了讨论。