Hive INSERT OVERWRITE显示错误

时间:2016-06-13 09:50:10

标签: java hadoop hive hbase bigdata

我正在研究将hbase-0.98.19与hive-1.2.1集成的示例。我使用命令

在hbase中创建了一个表
CREATE TABLE hbase_table_emp(id int, name string, role string) 
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ("hbase.columns.mapping" =     ":key,cf1:name,cf1:role")
 TBLPROPERTIES ("hbase.table.name" = "emp");

然后我创建了' testemp'用于将数据导入' hbase_table_emp'。以下代码显示了创建' testemp'表

create table testemp(id int, name string, role string) row format delimited fields terminated by '\t';
load data local inpath '/home/hduser/sample.txt' into table testemp;
select * from testemp;

直到现在,一切正常。但是当我运行命令insert overwrite table hbase_table_emp select * from testemp;

我收到以下错误: -

  

蜂房>从testemp插入覆盖表hbase_table_emp select *;   查询ID = hduser_20160613131557_ddef0b47-a773-477b-94d2-5cc070eb0de6   总工作数= 1启动工作1用于1个减少任务数量已设置   为0,因为没有reduce运算符java.io.IOException:   org.apache.hadoop.hive.ql.metadata.HiveException:   java.lang.IllegalArgumentException:必须在表中指定表名   org.apache.hadoop.hive.ql.exec.FileSinkOperator.checkOutputSpecs(FileSinkOperator.java:1117)     在   org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl.checkOutputSpecs(HiveOutputFormatImpl.java:67)     在   org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:564)     在   org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:432)     在org.apache.hadoop.mapreduce.Job $ 10.run(Job.java:1296)at at   org.apache.hadoop.mapreduce.Job $ 10.run(Job.java:1293)at at   java.security.AccessController.doPrivileged(Native Method)at   javax.security.auth.Subject.doAs(Subject.java:415)at   org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)     在org.apache.hadoop.mapreduce.Job.submit(Job.java:1293)at   org.apache.hadoop.mapred.JobClient $ 1.run(JobClient.java:562)at at   org.apache.hadoop.mapred.JobClient $ 1.run(JobClient.java:557)at at   java.security.AccessController.doPrivileged(Native Method)at   javax.security.auth.Subject.doAs(Subject.java:415)at   org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)     在   org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:557)     在org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:548)     在   org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:431)     在   org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(MapRedTask.java:137)     在org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160)at   org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:88)     在org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1653)at   org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1412)at   org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1195)at   org.apache.hadoop.hive.ql.Driver.run(Driver.java:1059)at at   org.apache.hadoop.hive.ql.Driver.run(Driver.java:1049)at at   org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:213)     在   org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:165)     在   org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:376)     在   org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:736)     在org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)at   org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)at at   sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at   sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)     在   sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)     在java.lang.reflect.Method.invoke(Method.java:606)at   org.apache.hadoop.util.RunJar.run(RunJar.java:221)at   org.apache.hadoop.util.RunJar.main(RunJar.java:136)引起:   org.apache.hadoop.hive.ql.metadata.HiveException:   java.lang.IllegalArgumentException:必须在表中指定表名   org.apache.hadoop.hive.ql.exec.FileSinkOperator.createHiveOutputFormat(FileSinkOperator.java:1139)     在   org.apache.hadoop.hive.ql.exec.FileSinkOperator.checkOutputSpecs(FileSinkOperator.java:1114)     ... 37更多引起:java.lang.IllegalArgumentException:必须   在表中指定表名   org.apache.hadoop.hbase.mapreduce.TableOutputFormat.setConf(TableOutputFormat.java:193)     在   org.apache.hive.common.util.ReflectionUtil.setConf(ReflectionUtil.java:101)     在   org.apache.hive.common.util.ReflectionUtil.newInstance(ReflectionUtil.java:87)     在   org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getHiveOutputFormat(HiveFileFormatUtils.java:277)     在   org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getHiveOutputFormat(HiveFileFormatUtils.java:267)     在   org.apache.hadoop.hive.ql.exec.FileSinkOperator.createHiveOutputFormat(FileSinkOperator.java:1137)     ... 38个工作提交失败,但有例外   '产生java.io.IOException(org.apache.hadoop.hive.ql.metadata.HiveException:   java.lang.IllegalArgumentException:必须指定表名)'失败:   执行错误,返回代码1   org.apache.hadoop.hive.ql.exec.mr.MapRedTask`

PS:我在CLASSPATH中包含了hbase.jar,zookeeper.jar和guava.jar。

提前致谢。

1 个答案:

答案 0 :(得分:0)

对于Hive HBase集成,为了将数据插入到hbase表中,还需要在TBLPROPERTIES中指定hbase.mapred.output.outputtable。

Hive HBase Integration

  

hbase.mapred.output.outputtable属性是可选的;这需要   如果您打算将数据插入表中(该属性由...使用)   hbase.mapreduce.TableOutputFormat)

因此,对于您的表,您需要运行以下命令:

ALTER TABLE table_name SET TBLPROPERTIES ("hbase.mapred.output.outputtable" = "emp");