Question

2018-12-14 17:41:57,313 [INFO] [InputInitializer {Map 1} #0] |io.HiveInputFormat|: Generating splits
2018-12-14 17:41:57,331 [INFO] [InputInitializer {Map 1} #0] |log.PerfLogger|: <PERFLOG method=OrcGetSplits from=org.apache.hadoop.hive.ql.io.orc.ReaderImpl>
2018-12-14 17:41:57,335 [INFO] [InputInitializer {Map 1} #0] |dag.RootInputInitializerManager|: Failed InputInitializer for Input: t1 on vertex vertex_1535529560012_804349_1_02 [Map 1]
2018-12-14 17:41:57,336 [ERROR] [Dispatcher thread {Central}] |impl.VertexImpl|: Vertex Input: t1 initializer failed, vertex=vertex_1535529560012_804349_1_02 [Map 1]
org.apache.tez.dag.app.dag.impl.AMUserCodeException: java.lang.ArrayIndexOutOfBoundsException: 1
    at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallback.onFailure(RootInputInitializerManager.java:327)
    at com.google.common.util.concurrent.Futures$6.run(Futures.java:977)
    at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253)
    at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149)
    at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:134)
    at com.google.common.util.concurrent.ListenableFutureTask.done(ListenableFutureTask.java:86)
    at java.util.concurrent.FutureTask.finishCompletion(FutureTask.java:384)
    at java.util.concurrent.FutureTask.setException(FutureTask.java:251)
    at java.util.concurrent.FutureTask.run(FutureTask.java:271)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ArrayIndexOutOfBoundsException: 1
    at org.apache.hadoop.hive.common.ValidReadTxnList.readFromString(ValidReadTxnList.java:142)
    at org.apache.hadoop.hive.common.ValidReadTxnList.<init>(ValidReadTxnList.java:57)
    at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$Context.<init>(OrcInputFormat.java:413)
    at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.generateSplitsInfo(OrcInputFormat.java:941)
    at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getSplits(OrcInputFormat.java:971)
    at org.apache.hadoop.hive.ql.io.HiveInputFormat.addSplitsForGroup(HiveInputFormat.java:306)
    at org.apache.hadoop.hive.ql.io.HiveInputFormat.getSplits(HiveInputFormat.java:408)
    at org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.initialize(HiveSplitGenerator.java:130)
    at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:278)
    at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:269)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1920)
    at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:269)
    at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:253)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    ... 3 more

Answer 1

问题已解决，这是类冲突。

我将hive-storage-api-2.6.0.jar导入Hive-common-1.1.0-cdh5.11.0.jar到一个pom.xml中。 hive-storage-api-2.6.0.jar中的 org.apache.hadoop.hive.common.ValidReadTxnList 与...发生冲突 org.apache.hadoop.hive.common.ValidReadTxnList Hive-common-1.1.0-cdh5.11.0.jar 。

当我在蜂巢使用执行引擎tez上运行任务时，发生错误

1 个答案: