Question

当我尝试在终端中运行TEZ作业时，我的hive崩溃了。我在一个有5个节点的集群上安装了HDP 2.1。操作系统：Redhat Enterprise

谁能帮帮我？：）

从应用程序跟踪器记录：

HDP: 2.1
User: hdfs
Name: HIVE-9fd632e5-d76a-4e0a-8845-0c1f0af52f5c
Application Type: TEZ
Application Tags:
State: FAILED
FinalStatus: FAILED
Started: 1-Jul-2014 12:03:22
Elapsed: 4sec
Tracking URL: History
Diagnostics:
Application application_1404208879507_0002 failed 2 times due to AM Container for    appattempt_1404208879507_0002_000002 exited with exitCode: 1 due to: Exception from container-    launch:     org.apache.hadoop.util.Shell$ExitCodeException:
org.apache.hadoop.util.Shell$ExitCodeException:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:505)
at org.apache.hadoop.util.Shell.run(Shell.java:418)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:650)
at         org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecu    tor.java:195)
at     org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch    .java:300)
at     org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch    .java:81)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Container exited with a non-zero exit code 1
.Failing this attempt.. Failing the application.

在终端：

Total jobs = 1
Launching Job 1 out of 1
Tez session was closed. Reopening...
Session re-established.
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask

Answer 1

原因可能是运行作业的用户的ID不能小于1000.转到TaskTracker的日志文件以查看是否有如下消息：

INFO org.apache.hadoop.mapred.TaskController: Reading task controller configuration /etc/hadoop/taskcontroller.cfg
INFO. org.apache.hadoop.mapred.TaskController: requested user
hdfs has id 201, which is below the minimum allowed 1000

如果是这种情况，那么您可以执行以下操作：

usermod -u 10000 hdfs

Answer 2

我认为Tez非常不稳定。只选择* from from from where clausules目前正在工作..我在Hue上以HDFS身份登录并从...查询中激活了一个正常的select *。谢谢你的快速回答！

Answer 3

由于AM Container for“应用程序application_1404208879507_0002失败2次”这一行表明Tez Application Master可能无法启动。这可能是由于各种原因造成的。

查找原因的最简单的地方是通过调用“bin / yarn logs -applicationId application_1404208879507_0002”获得的应用程序日志。导致这种情况的最常见问题通常是在启动java进程时导致ClassNotFound错误的一些设置/配置错误。

Tez在HDP 2.1上崩溃

3 个答案: