我们计划通过Spring-XD执行sqoop作业,并按照Spring-XD文档中的建议进行配置。
以下是servers.xml文件中的配置
spring: hadoop: fsUri: hdfs://nnservice resourceManagerHost: server-m01.mydomain.com resourceManagerPort: 8032 resourceManagerSchedulerAddress: ${spring.hadoop.resourceManagerHost}:8030 jobHistoryAddress: server-m02.mydomain.com:10020 security: authMethod: kerberos userPrincipal: springxd@mydomain.COM userKeytab: /home/springxd/springxd.keytab namenodePrincipal: nn/_HOST@mydomain.COM rmManagerPrincipal: rm/_HOST@mydomain.COM config: mapreduce.framework.name: yarn dfs.nameservices: nnservice dfs.ha.namenodes.nnservice: nn1, nn2 dfs.namenode.rpc-address.nnservice.nn1: server-m01.mydomain.com:8020 dfs.namenode.rpc-address.nnservice.nn2: server-m02.mydomain.com:8020 dfs.client.failover.proxy.provider.nnservice: org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider dfs.ha.automatic-failover.enabled: True mapreduce.application.framework.path: '/hdp/apps/2.4.2.0-258/mapreduce/mapreduce.tar.gz#mr-framework' mapreduce.application.classpath: '$PWD/mr-framework/hadoop/share/hadoop/mapreduce/*:$PWD/mr-framework/hadoop/share/hadoop/mapreduce/lib/*:$PWD/mr-framework/hadoop/share/hadoop/common/*:$PWD/mr-framework/hadoop/share/hadoop/common/lib/*:$PWD/mr-framework/hadoop/share/hadoop/yarn/*:$PWD/mr-framework/hadoop/share/hadoop/yarn/lib/*:$PWD/mr-framework/hadoop/share/hadoop/hdfs/*:$PWD/mr-framework/hadoop/share/hadoop/hdfs/lib/*:/usr/hdp/2.4.2.0-258/hadoop/lib/hadoop-lzo-0.6.0.2.4.2.0-258.jar:/etc/hadoop/conf/secure' yarnApplicationClasspath: '$HADOOP_CONF_DIR,/usr/hdp/current/hadoop-client/*,/usr/hdp/current/hadoop-client/lib/*,/usr/hdp/current/hadoop-hdfs-client/*,/usr/hdp/current/hadoop-hdfs-client/lib/*,/usr/hdp/current/hadoop-yarn-client/*,/usr/hdp/current/hadoop-yarn-client/lib/*'
sqoop作业因以下错误而终止
2016-10-28T10:08:35-0400 1.3.1.RELEASE INFO main mapreduce.Job - Job job_1477527264038_0051 failed with state FAILED due to: Application application_1477527264038_0051 failed 2 times due to AM Container for appattempt_1477527264038_0051_000002 exited with exitCode: 1
知道我们是否缺少任何配置?