Apache Kylin:Cube构建在第3步失败

时间:2017-03-14 05:32:42

标签: hadoop kylin

我正在尝试构建一个Apache Kylin多维数据集,但在步骤3中失败了以下log msg:

  

java.lang.RuntimeException:java.io.IOException:NoSuchObjectException(message:default.kylin_intermediate_t3cube_47ec8760_55ce_445d_a91b_0c33957690bf table not found)       在org.apache.kylin.source.hive.HiveMRInput $ HiveTableInputFormat.configureJob(HiveMRInput.java:110)       在org.apache.kylin.engine.mr.steps.FactDistinctColumnsJob.setupMapper(FactDistinctColumnsJob.java:119)       在org.apache.kylin.engine.mr.steps.FactDistinctColumnsJob.run(FactDistinctColumnsJob.java:103)       在org.apache.kylin.engine.mr.MRUtil.runMRJob(MRUtil.java:92)       在org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:120)       在org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:113)       在org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57)       在org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:113)       在org.apache.kylin.job.impl.threadpool.DefaultScheduler $ JobRunner.run(DefaultScheduler.java:136)       在java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)       at java.util.concurrent.ThreadPoolExecutor $ Worker.run(ThreadPoolExecutor.java:617)       在java.lang.Thread.run(Thread.java:745)   引起:java.io.IOException:NoSuchObjectException(消息:default.kylin_intermediate_t3cube_47ec8760_55ce_445d_a91b_0c33957690bf表未找到)       at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:97)       at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:51)       在org.apache.kylin.source.hive.HiveMRInput $ HiveTableInputFormat.configureJob(HiveMRInput.java:105)       ......还有11个   引起:NoSuchObjectException(消息:default.kylin_intermediate_t3cube_47ec8760_55ce_445d_a91b_0c33957690bf表未找到)       在org.apache.hadoop.hive.metastore.HiveMetaStore $ HMSHandler.get_table_core(HiveMetaStore.java:1946)       在org.apache.hadoop.hive.metastore.HiveMetaStore $ HMSHandler.get_table(HiveMetaStore.java:1899)       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)       at java.lang.reflect.Method.invoke(Method.java:498)       在org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:140)       在org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:99)       at com.sun.proxy。$ Proxy47.get_table(Unknown Source)       在org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:1332)       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)       at java.lang.reflect.Method.invoke(Method.java:498)       在org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:150)       在com.sun.proxy。$ Proxy48.getTable(未知来源)       在org.apache.hive.hcatalog.common.HCatUtil.getTable(HCatUtil.java:180)       at org.apache.hive.hcatalog.mapreduce.InitializeInput.getInputJobInfo(InitializeInput.java:105)       在org.apache.hive.hcatalog.mapreduce.InitializeInput.setInput(InitializeInput.java:88)       at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:95)       ......还有13个     结果代码:2

该消息显示kylin找不到表“default.kylin_intermediate_t3cube_47ec8760_55ce_445d_a91b_0c33957690bf”,但是当我登录到hive时,该表存在。

有人可以帮我解决一下吗?

编辑:   我已经解决了这个问题。 Kylin使用HCatelog来获取表信息,但我没有正确配置hive。

1 个答案:

答案 0 :(得分:1)

Kylin通过hcatalog lib直接连接到hive Metastore。这需要类路径上的hive-site.xml,并且它必须包含足够的信息(例如' hive.metastore.uris')以连接到正确的hive Metastore。否则,如果连接到错误的Metastore,将发生这样的无表发现异常。

检查失败的" Extract Fact Table Distinct Columns"的MR配置。 YARN资源管理器或作业历史记录服务器中的作业。搜索类似路径属性,例如' mapreduce.application.classpath'。检查hive-site.xml是否在类路径上,它包含正确的Metastore连接信息。

注意" $ KYLIN_HOME / conf / kylin_job_conf.xml"您可以用来覆盖任何MR作业配置,例如' mapreduce.application.classpath'。

KYLIN-2530是一项改进措施。假设这个问题将从Kylin 2.0开始缓解。