Question

我在HDP2.2上使用Hive14，并且在Hive上有索引问题。我可以创建一个索引。

create INDEX ix_key ON TABLE DbTest.Tbl_test(TEST_KEY)
as 'org.apache.hadoop.hive.ql.index.compact.CompactIndexHandler' WITH DEFERRED REBUILD;

之后我将数据加载到表中并构建索引。

ALTER INDEX ix_key ON DbTest.Tbl_test REBUILD;

Hive构建了索引并且工作正常，性能提升了。现在我想重建索引，但总是出错：

INFO  : Session is already open
INFO  : Tez session was closed. Reopening...
INFO  : Session re-established.
INFO  : 

ERROR : Status: Failed
ERROR : Vertex failed, vertexName=Map 1, vertexId=vertex_1426585957958_2810_1_00, diagnostics=[Vertex vertex_1426585957958_2810_1_00 [Map 1] killed/failed due to:ROOT_INPUT_INIT_FAILURE, Vertex Input: Tbl_test initializer failed, vertex=vertex_1426585957958_2810_1_00 [Map 1], java.lang.NullPointerException
at org.apache.hadoop.hive.ql.exec.tez.DynamicPartitionPruner.initialize(DynamicPartitionPruner.java:135)
at org.apache.hadoop.hive.ql.exec.tez.DynamicPartitionPruner.prune(DynamicPartitionPruner.java:100)
at org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.initialize(HiveSplitGenerator.java:109)
at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:245)
at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:239)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:239)
at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:226)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]
ERROR : Vertex killed, vertexName=Reducer 2, vertexId=vertex_1426585957958_2810_1_01, diagnostics=[Vertex received Kill in INITED state., Vertex vertex_1426585957958_2810_1_01 [Reducer 2] killed/failed due to:null]
ERROR : DAG failed due to vertex failure. failedVertices:1 killedVertices:1
Error: Error while processing statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask (state=08S01,code=2)

基表存在，我可以对它运行查询。索引表也存在。如果我在另一个表上创建一个新索引并运行rebuild-command，我会得到相同的错误。我用直线和CLI尝试了它 - 错误始终是一样的。

希望任何人都有一些想法如何解决这个问题。

Answer 1

这似乎在动态分区修剪程序中失败，您可以使用＆＃34; hive.tez.dynamic.partition.pruning = false＆＃34;关闭它。您也可以考虑在此处提交错误：https://issues.apache.org

Hive Index不能REBUILD - DAG在顶点失败时失败

1 个答案: