HortonWorks版本是2.4 SpringXd版本 - 1.3.1
我正在尝试运行一个sqoop作业,以便从Spring XD导入。它提供以下堆栈跟踪。你能告诉我这里的问题吗?
2016-05-16T11:29:54+0530 1.3.1.RELEASE INFO main tool.CodeGenTool - Beginning code generation
2016-05-16T11:29:54+0530 1.3.1.RELEASE INFO main manager.SqlManager - Executing SQL statement: SELECT t.* FROM Events AS t WHERE 1=0
2016-05-16T11:29:54+0530 1.3.1.RELEASE INFO main manager.SqlManager - Executing SQL statement: SELECT t.* FROM Events AS t WHERE 1=0
2016-05-16T11:29:54+0530 1.3.1.RELEASE INFO main orm.CompilationManager - HADOOP_MAPRED_HOME is /Users/nxjain/Documents/spring-xd-1.3.1.RELEASE/xd/lib/hdp22
2016-05-16T11:29:56+0530 1.3.1.RELEASE INFO main orm.CompilationManager - Writing jar file: /tmp/sqoop-nxjain/compile/855df3a42e72c92f2a5d87ffc3827e48/Events.jar
2016-05-16T11:29:56+0530 1.3.1.RELEASE INFO main mapreduce.ImportJobBase - Beginning import of Events
2016-05-16T11:29:57+0530 1.3.1.RELEASE WARN main util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2016-05-16T11:29:57+0530 1.3.1.RELEASE INFO main manager.SqlManager - Executing SQL statement: SELECT t.* FROM Events AS t WHERE 1=0
2016-05-16T11:29:57+0530 1.3.1.RELEASE WARN main mapreduce.JobBase - SQOOP_HOME is unset. May not be able to find all job dependencies.
2016-05-16T11:29:57+0530 1.3.1.RELEASE INFO main client.RMProxy - Connecting to ResourceManager at /172.16.128.128:8050
2016-05-16T11:29:58+0530 1.3.1.RELEASE INFO main db.DBInputFormat - Using read commited transaction isolation
2016-05-16T11:29:58+0530 1.3.1.RELEASE INFO main mapreduce.JobSubmitter - number of splits:1
2016-05-16T11:29:59+0530 1.3.1.RELEASE INFO main mapreduce.JobSubmitter - Submitting tokens for job: job_1463375311370_0002
2016-05-16T11:29:59+0530 1.3.1.RELEASE INFO main impl.YarnClientImpl - Submitted application application_1463375311370_0002
2016-05-16T11:29:59+0530 1.3.1.RELEASE INFO main mapreduce.Job - The url to track the job: http://sandbox.hortonworks.com:8088/proxy/application_1463375311370_0002/
2016-05-16T11:29:59+0530 1.3.1.RELEASE INFO main mapreduce.Job - Running job: job_1463375311370_0002
2016-05-16T11:30:03+0530 1.3.1.RELEASE INFO main mapreduce.Job - Job job_1463375311370_0002 running in uber mode : false
2016-05-16T11:30:03+0530 1.3.1.RELEASE INFO main mapreduce.Job - map 0% reduce 0%
2016-05-16T11:30:03+0530 1.3.1.RELEASE INFO main mapreduce.Job - Job job_1463375311370_0002 failed with state FAILED due to: Application application_1463375311370_0002 failed 2 times due to AM Container for appattempt_1463375311370_0002_000002 exited with exitCode: 1
For more detailed output, check application tracking page:http://sandbox.hortonworks.com:8088/cluster/app/application_1463375311370_0002Then, click on links to logs of each attempt.
Diagnostics: Exception from container-launch.
Container id: container_e09_1463375311370_0002_02_000001
Exit code: 1
Stack trace: ExitCodeException exitCode=1:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:576)
at org.apache.hadoop.util.Shell.run(Shell.java:487)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:753)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:212)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:303)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Container exited with a non-zero exit code 1
Failing this attempt. Failing the application.
2016-05-16T11:30:03+0530 1.3.1.RELEASE INFO main mapreduce.Job - Counters: 0
2016-05-16T11:30:03+0530 1.3.1.RELEASE WARN main mapreduce.Counters - Group FileSystemCounters is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead
2016-05-16T11:30:03+0530 1.3.1.RELEASE INFO main mapreduce.ImportJobBase - Transferred 0 bytes in 5.7269 seconds (0 bytes/sec)
2016-05-16T11:30:03+0530 1.3.1.RELEASE WARN main mapreduce.Counters - Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead
2016-05-16T11:30:03+0530 1.3.1.RELEASE INFO main mapreduce.ImportJobBase - Retrieved 0 records.
2016-05-16T11:30:03+0530 1.3.1.RELEASE ERROR main tool.ImportTool - Error during import: Import job failed!
答案 0 :(得分:0)
这很可能是由于类路径问题 - 您可以检查失败的YARN容器的日志。
看一下这条评论:https://github.com/spring-projects/spring-xd/pull/1451#issue-56883941
我们为正在使用的HDP版本添加了一些特定的配置设置。您必须根据您使用的版本和构建进行调整。请注意,我们还没有使用HDP 2.4测试XD。它可能有用,所以让我们知道你是如何做出来的。