在Apache Kylin中构建多维数据集时出现错误

时间：2019-06-24 21:47:30

标签： java scala hive olap kylin

请参阅第一步中的日志：

java.io.IOException: OS command error exit with return code: 127, error message: 
Logging initialized using configuration in jar:file:/home/jefferson/Developer/apache-hive-1.2.2/lib/hive-common-1.2.2.jar!/hive-log4j.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/jefferson/Developer/hbase-1.4.9/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/jefferson/Developer/hadoop-2.7.7/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
OK
Time taken: 0.634 seconds
OK
Time taken: 0.06 seconds
OK
Time taken: 0.531 seconds
OK
Time taken: 0.163 seconds
Query ID = root_20190624182158_03e9235b-de36-4498-b977-f5d343c4807c
Total jobs = 1
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/jefferson/Developer/hbase-1.4.9/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/jefferson/Developer/hadoop-2.7.7/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Execution log at: /tmp/root/root_20190624182158_03e9235b-de36-4498-b977-f5d343c4807c.log
2019-06-24 18:22:01 Starting to launch local task to process map join;  maximum memory = 3817865216
2019-06-24 18:22:01 Dump the side-table for tag: 1 with group count: 244 into file: file:/tmp/root/fce0b7ff-ed96-4bc6-bb85-5935076c99aa/hive_2019-06-24_18-21-58_050_2107942375316508204-1/-local-10007/HashTable-Stage-14/MapJoin-mapfile01--.hashtable
2019-06-24 18:22:02 Uploaded 1 File to: file:/tmp/root/fce0b7ff-ed96-4bc6-bb85-5935076c99aa/hive_2019-06-24_18-21-58_050_2107942375316508204-1/-local-10007/HashTable-Stage-14/MapJoin-mapfile01--.hashtable (5163 bytes)
2019-06-24 18:22:02 Dump the side-table for tag: 1 with group count: 244 into file: file:/tmp/root/fce0b7ff-ed96-4bc6-bb85-5935076c99aa/hive_2019-06-24_18-21-58_050_2107942375316508204-1/-local-10007/HashTable-Stage-14/MapJoin-mapfile11--.hashtable
2019-06-24 18:22:02 Uploaded 1 File to: file:/tmp/root/fce0b7ff-ed96-4bc6-bb85-5935076c99aa/hive_2019-06-24_18-21-58_050_2107942375316508204-1/-local-10007/HashTable-Stage-14/MapJoin-mapfile11--.hashtable (5163 bytes)
2019-06-24 18:22:02 Dump the side-table for tag: 1 with group count: 10000 into file: file:/tmp/root/fce0b7ff-ed96-4bc6-bb85-5935076c99aa/hive_2019-06-24_18-21-58_050_2107942375316508204-1/-local-10007/HashTable-Stage-14/MapJoin-mapfile21--.hashtable
2019-06-24 18:22:02 Uploaded 1 File to: file:/tmp/root/fce0b7ff-ed96-4bc6-bb85-5935076c99aa/hive_2019-06-24_18-21-58_050_2107942375316508204-1/-local-10007/HashTable-Stage-14/MapJoin-mapfile21--.hashtable (251480 bytes)
2019-06-24 18:22:02 Dump the side-table for tag: 1 with group count: 10000 into file: file:/tmp/root/fce0b7ff-ed96-4bc6-bb85-5935076c99aa/hive_2019-06-24_18-21-58_050_2107942375316508204-1/-local-10007/HashTable-Stage-14/MapJoin-mapfile31--.hashtable
2019-06-24 18:22:02 Uploaded 1 File to: file:/tmp/root/fce0b7ff-ed96-4bc6-bb85-5935076c99aa/hive_2019-06-24_18-21-58_050_2107942375316508204-1/-local-10007/HashTable-Stage-14/MapJoin-mapfile31--.hashtable (251480 bytes)
2019-06-24 18:22:02 Dump the side-table for tag: 1 with group count: 144 into file: file:/tmp/root/fce0b7ff-ed96-4bc6-bb85-5935076c99aa/hive_2019-06-24_18-21-58_050_2107942375316508204-1/-local-10007/HashTable-Stage-14/MapJoin-mapfile41--.hashtable
2019-06-24 18:22:02 Uploaded 1 File to: file:/tmp/root/fce0b7ff-ed96-4bc6-bb85-5935076c99aa/hive_2019-06-24_18-21-58_050_2107942375316508204-1/-local-10007/HashTable-Stage-14/MapJoin-mapfile41--.hashtable (3348 bytes)
2019-06-24 18:22:02 Dump the side-table for tag: 1 with group count: 0 into file: file:/tmp/root/fce0b7ff-ed96-4bc6-bb85-5935076c99aa/hive_2019-06-24_18-21-58_050_2107942375316508204-1/-local-10007/HashTable-Stage-14/MapJoin-mapfile51--.hashtable
2019-06-24 18:22:02 Uploaded 1 File to: file:/tmp/root/fce0b7ff-ed96-4bc6-bb85-5935076c99aa/hive_2019-06-24_18-21-58_050_2107942375316508204-1/-local-10007/HashTable-Stage-14/MapJoin-mapfile51--.hashtable (260 bytes)
2019-06-24 18:22:02 End of local task; Time Taken: 0.944 sec.
Execution completed successfully
MapredLocal task succeeded
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Job running in-process (local Hadoop)
2019-06-24 18:22:04,049 Stage-14 map = 100%,  reduce = 0%, Cumulative CPU 1.84 sec
MapReduce Total cumulative CPU time: 1 seconds 840 msec
Ended Job = job_local1579904003_0001
Loading data to table default.kylin_intermediate_relatorio_por_paises_a1675df9_2dfc_3d8c_cd70_245f440477cb
Table default.kylin_intermediate_relatorio_por_paises_a1675df9_2dfc_3d8c_cd70_245f440477cb stats: [numFiles=0, numRows=0, totalSize=0, rawDataSize=0]
MapReduce Jobs Launched: 
Stage-Stage-14:  Cumulative CPU: 1.84 sec   HDFS Read: 798009 HDFS Write: 242 SUCCESS
Total MapReduce CPU Time Spent: 1 seconds 840 msec
OK
Time taken: 6.207 seconds
/bin/bash: line 32: --hiveconf: command not found
The command is: 
hive -e "USE default;

DROP TABLE IF EXISTS kylin_intermediate_relatorio_por_paises_a1675df9_2dfc_3d8c_cd70_245f440477cb;
CREATE EXTERNAL TABLE IF NOT EXISTS kylin_intermediate_relatorio_por_paises_a1675df9_2dfc_3d8c_cd70_245f440477cb
(
BUYER_ACCOUNT_ACCOUNT_COUNTRY string
,KYLIN_SALES_PRICE decimal(19,4)
,KYLIN_SALES_ITEM_COUNT bigint
)
STORED AS SEQUENCEFILE
LOCATION 'hdfs://localhost:8020/home/jefferson/kylin/kylin_metadata/kylin-47e822a4-9f9f-c19f-8ede-c125052e2fb4/kylin_intermediate_relatorio_por_paises_a1675df9_2dfc_3d8c_cd70_245f440477cb';
ALTER TABLE kylin_intermediate_relatorio_por_paises_a1675df9_2dfc_3d8c_cd70_245f440477cb SET TBLPROPERTIES('auto.purge'='true');
INSERT OVERWRITE TABLE \`kylin_intermediate_relatorio_por_paises_a1675df9_2dfc_3d8c_cd70_245f440477cb\` SELECT
\`BUYER_ACCOUNT\`.\`ACCOUNT_COUNTRY\` as \`BUYER_ACCOUNT_ACCOUNT_COUNTRY\`
,\`KYLIN_SALES\`.\`PRICE\` as \`KYLIN_SALES_PRICE\`
,\`KYLIN_SALES\`.\`ITEM_COUNT\` as \`KYLIN_SALES_ITEM_COUNT\`
 FROM \`DEFAULT\`.\`KYLIN_SALES\` as \`KYLIN_SALES\`
INNER JOIN \`DEFAULT\`.\`KYLIN_CAL_DT\` as \`KYLIN_CAL_DT\`
ON \`KYLIN_SALES\`.\`PART_DT\` = \`KYLIN_CAL_DT\`.\`CAL_DT\`
INNER JOIN \`DEFAULT\`.\`KYLIN_CATEGORY_GROUPINGS\` as \`KYLIN_CATEGORY_GROUPINGS\`
ON \`KYLIN_SALES\`.\`LEAF_CATEG_ID\` = \`KYLIN_CATEGORY_GROUPINGS\`.\`LEAF_CATEG_ID\` AND \`KYLIN_SALES\`.\`LSTG_SITE_ID\` = \`KYLIN_CATEGORY_GROUPINGS\`.\`SITE_ID\`
INNER JOIN \`DEFAULT\`.\`KYLIN_ACCOUNT\` as \`BUYER_ACCOUNT\`
ON \`KYLIN_SALES\`.\`BUYER_ID\` = \`BUYER_ACCOUNT\`.\`ACCOUNT_ID\`
INNER JOIN \`DEFAULT\`.\`KYLIN_ACCOUNT\` as \`SELLER_ACCOUNT\`
ON \`KYLIN_SALES\`.\`SELLER_ID\` = \`SELLER_ACCOUNT\`.\`ACCOUNT_ID\`
INNER JOIN \`DEFAULT\`.\`KYLIN_COUNTRY\` as \`BUYER_COUNTRY\`
ON \`BUYER_ACCOUNT\`.\`ACCOUNT_COUNTRY\` = \`BUYER_COUNTRY\`.\`COUNTRY\`
INNER JOIN \`DEFAULT\`.\`KYLIN_COUNTRY\` as \`SELLER_COUNTRY\`
ON \`SELLER_ACCOUNT\`.\`ACCOUNT_COUNTRY\` = \`SELLER_COUNTRY\`.\`COUNTRY\`
WHERE 1=1 AND (\`KYLIN_SALES\`.PART_DT >= '2019-06-24' AND \`KYLIN_SALES\`.PART_DT < '2019-06-24')
;

" --hiveconf hive.merge.mapredfiles=false --hiveconf hive.metastore.schema.verification=true --hiveconf hive.auto.convert.join=true --hiveconf dfs.replication=2 --hiveconf hive.exec.compress.output=true --hiveconf hive.auto.convert.join.noconditionaltask=true --hiveconf mapreduce.job.split.metainfo.maxsize=-1 --hiveconf javax.jdo.option.ConnectionURL=jdbc:derby:/home/jefferson/metastore_db;create=true --hiveconf hive.merge.mapfiles=false --hiveconf hive.auto.convert.join.noconditionaltask.size=100000000 --hiveconf hive.stats.autogather=true
    at org.apache.kylin.common.util.CliCommandExecutor.execute(CliCommandExecutor.java:96)
    at org.apache.kylin.source.hive.CreateFlatHiveTableStep.createFlatHiveTable(CreateFlatHiveTableStep.java:62)
    at org.apache.kylin.source.hive.CreateFlatHiveTableStep.doWork(CreateFlatHiveTableStep.java:99)
    at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:166)
    at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
    at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:166)
    at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

1 个答案:

答案 0 :(得分：1)

在Kylin 2.6中，所有数据库名称，表名称和列名称必须为大写。将它们全部更改为大写，您的错误将得到解决。

相关问题
最新问题