Hive LLAP守护程序不适用于ODBC / JDBC

时间:2018-09-27 17:46:45

标签: jdbc hive odbc hadoop2 amazon-emr

我已通过Slider在群集上部署了Hive LLAP。 Hive提示符显然通过搭接提交作业。当我尝试通过odbc连接在beeline中运行相同的查询(实际上是任何查询)时,出现错误:

0: jdbc:hive2://localhost:10000/default> select count(1) from testtable limit 1;
ERROR : Status: Failed
ERROR : Dag received [DAG_TERMINATE, SERVICE_PLUGIN_ERROR] in RUNNING state.
ERROR : Error reported by TaskScheduler [[2:LLAP]][SERVICE_UNAVAILABLE] No LLAP Daemons are running
ERROR : Vertex killed, vertexName=Reducer 2, vertexId=vertex_1538061470561_0009_2_01, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to DAG_TERMINATED, failedTasks:0 killedTasks:1, Vertex vertex_1538061470561_0009_2_01 [Reducer 2] killed/failed due to:DAG_TERMINATED]
ERROR : Vertex killed, vertexName=Map 1, vertexId=vertex_1538061470561_0009_2_00, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to DAG_TERMINATED, failedTasks:0 killedTasks:1, Vertex vertex_1538061470561_0009_2_00 [Map 1] killed/failed due to:DAG_TERMINATED]
ERROR : DAG did not succeed due to SERVICE_PLUGIN_ERROR. failedVertices:0 killedVertices:2
INFO  : org.apache.tez.common.counters.DAGCounter:
INFO  :    NUM_KILLED_TASKS: 1
INFO  :    AM_CPU_MILLISECONDS: 1380
INFO  :    AM_GC_TIME_MILLIS: 0
ERROR : FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Dag received [DAG_TERMINATE, SERVICE_PLUGIN_ERROR] in RUNNING state.Error reported by TaskScheduler [[2:LLAP]][SERVICE_UNAVAILABLE] No LLAP Daemons are runningVertex killed, vertexName=Reducer 2, vertexId=vertex_1538061470561_0009_2_01, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to DAG_TERMINATED, failedTasks:0 killedTasks:1, Vertex vertex_1538061470561_0009_2_01 [Reducer 2] killed/failed due to:DAG_TERMINATED]Vertex killed, vertexName=Map 1, vertexId=vertex_1538061470561_0009_2_00, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to DAG_TERMINATED, failedTasks:0 killedTasks:1, Vertex vertex_1538061470561_0009_2_00 [Map 1] killed/failed due to:DAG_TERMINATED]DAG did not succeed due to SERVICE_PLUGIN_ERROR. failedVertices:0 killedVertices:2
INFO  : Completed executing command(queryId=hive_20180927173233_63d898d4-c137-495d-91de-72482212ce43); Time taken: 60.516 seconds
Error: org.apache.hive.service.cli.HiveSQLException: Error while processing statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Dag received [DAG_TERMINATE, SERVICE_PLUGIN_ERROR] in RUNNING state.Error reported by TaskScheduler [[2:LLAP]][SERVICE_UNAVAILABLE] No LLAP Daemons are runningVertex killed, vertexName=Reducer 2, vertexId=vertex_1538061470561_0009_2_01, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to DAG_TERMINATED, failedTasks:0 killedTasks:1, Vertex vertex_1538061470561_0009_2_01 [Reducer 2] killed/failed due to:DAG_TERMINATED]Vertex killed, vertexName=Map 1, vertexId=vertex_1538061470561_0009_2_00, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to DAG_TERMINATED, failedTasks:0 killedTasks:1, Vertex vertex_1538061470561_0009_2_00 [Map 1] killed/failed due to:DAG_TERMINATED]DAG did not succeed due to SERVICE_PLUGIN_ERROR. failedVertices:0 killedVertices:2

Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Dag received [DAG_TERMINATE, SERVICE_PLUGIN_ERROR] in RUNNING state.Error reported by TaskScheduler [[2:LLAP]][SERVICE_UNAVAILABLE] No LLAP Daemons are runningVertex killed, vertexName=Reducer 2, vertexId=vertex_1538061470561_0009_2_01, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to DAG_TERMINATED, failedTasks:0 killedTasks:1, Vertex vertex_1538061470561_0009_2_01 [Reducer 2] killed/failed due to:DAG_TERMINATED]Vertex killed, vertexName=Map 1, vertexId=vertex_1538061470561_0009_2_00, diagnostics=[Vertex received Kill while in RUNNING state.,Vertex did not succeed due to DAG_TERMINATED, failedTasks:0 killedTasks:1, Vertex vertex_1538061470561_0009_2_00 [Map 1] killed/failed due to:DAG_TERMINATED]DAG did not succeed due to SERVICE_PLUGIN_ERROR. failedVertices:0 killedVertices:2

但是我知道守护进程实际上正在运行。我应该使用不同的配置来使通过ODBC运行的会话访问llap守护程序吗?

1 个答案:

答案 0 :(得分:0)

我了解到JDBC / ODBC用户默认以hive Unix用户身份运行。为了访问我的llap守护程序,我需要以hive用户身份运行滑块应用程序。以下调用可以达到目的:

sudo -H -u hive bash -c "${SLIDER_APP}/run.sh"