使用importtsv将文本文件导入HBase

时间：2013-01-20 03:46:17

标签： hadoop import hbase

问题：

我正在尝试使用文档中的this示例从文本文件将数据导入HBase数据库。

配置

HBase版本：hbase-0.94.4。

我的安装是Standalone（非分布式或伪分布式）。

重现错误

最初它在HDFS上创建StoreFiles，并使用名称datatsv创建一个名为HADOOP_CLASSPATH=`${HBASE_HOME}/bin/hbase classpath` ${HADOOP_HOME}/bin/hadoop \ jar /usr/local/hbase/hbase-0.94.4.jar importtsv \ -Dimporttsv.columns=HBASE_ROW_KEY,d:c1,d:c2 \ -Dimporttsv.bulk.output=/user/vm1/outputfile datatsv /user/vm1/myfile.txt 的空表：

hbase org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles \
/user/vm1/outputfile datatsv

然后，CompleteBulkLoad方法将数据从StoreFile加载到表：

HADOOP_CLASSPATH=`${HBASE_HOME}/bin/hbase classpath` ${HADOOP_HOME}/bin/hadoop \
jar /usr/local/hbase/hbase-0.94.4.jar completebulkload \
/user/vm1/outputfile datatsv

以下命令与上述命令具有相同的输出：

ERROR mapreduce.LoadIncrementalHFiles: Unexpected execution exception during splitting
java.util.concurrent.ExecutionException: java.lang.IllegalStateException: The value of the hbase.metrics.showTableName conf option has not been specified in SchemaMetrics
    at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252)
    at java.util.concurrent.FutureTask.get(FutureTask.java:111)
    at org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.groupOrSplitPhase(LoadIncrementalHFiles.java:333)
    at org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.doBulkLoad(LoadIncrementalHFiles.java:232)
    at org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.run(LoadIncrementalHFiles.java:699)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
    at org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.main(LoadIncrementalHFiles.java:704)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
    at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by: java.lang.IllegalStateException: The value of the hbase.metrics.showTableName conf option has not been specified in SchemaMetrics
    at org.apache.hadoop.hbase.regionserver.metrics.SchemaMetrics.getEffectiveTableName(SchemaMetrics.java:607)
    at org.apache.hadoop.hbase.regionserver.metrics.SchemaMetrics.getInstance(SchemaMetrics.java:333)
    at org.apache.hadoop.hbase.regionserver.metrics.SchemaConfigured.getSchemaMetrics(SchemaConfigured.java:185)
    at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.close(HFileReaderV2.java:441)
    at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.close(HFileReaderV2.java:419)
    at org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.groupOrSplit(LoadIncrementalHFiles.java:410)
    at org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$2.call(LoadIncrementalHFiles.java:323)
    at org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$2.call(LoadIncrementalHFiles.java:321)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
    at java.util.concurrent.FutureTask.run(FutureTask.java:166)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
    at java.lang.Thread.run(Thread.java:722)
Exception in thread "main" java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at org.apache.hadoop.hbase.mapreduce.Driver.main(Driver.java:51)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by: java.lang.IllegalStateException: java.lang.IllegalStateException: The value of the hbase.metrics.showTableName conf option has not been specified in SchemaMetrics
    at org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.groupOrSplitPhase(LoadIncrementalHFiles.java:344)
    at org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.doBulkLoad(LoadIncrementalHFiles.java:232)
    at org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.run(LoadIncrementalHFiles.java:699)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
    at org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.main(LoadIncrementalHFiles.java:704)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
    at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
    ... 10 more
Caused by: java.lang.IllegalStateException: The value of the hbase.metrics.showTableName conf option has not been specified in SchemaMetrics
    at org.apache.hadoop.hbase.regionserver.metrics.SchemaMetrics.getEffectiveTableName(SchemaMetrics.java:607)
    at org.apache.hadoop.hbase.regionserver.metrics.SchemaMetrics.getInstance(SchemaMetrics.java:333)
    at org.apache.hadoop.hbase.regionserver.metrics.SchemaConfigured.getSchemaMetrics(SchemaConfigured.java:185)
    at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.close(HFileReaderV2.java:441)
    at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.close(HFileReaderV2.java:419)
    at org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.groupOrSplit(LoadIncrementalHFiles.java:410)
    at org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$2.call(LoadIncrementalHFiles.java:323)
    at org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$2.call(LoadIncrementalHFiles.java:321)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
    at java.util.concurrent.FutureTask.run(FutureTask.java:166)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
    at java.lang.Thread.run(Thread.java:722)

任一命令的错误消息是：

hbase.metrics.showTableName

问题

我错过了什么吗？

搜索邮件列表，在免费节点上询问，但未找到答案。还尝试使用以下方法设置hbase的属性（并重新启动它）： false至{{1}}

任何帮助将不胜感激！ 谢谢！

2 个答案:

答案 0 :(得分：2)

当我尝试使用Java API批量加载数据时，我有一个非常类似的问题。我打开了一个帖子here。我必须在我的客户端应用程序中指定此行，这解决了问题：SchemaMetrics.configureGlobally(conf)。现在，您不使用Java API，所有这一行都是在.xml配置中设置属性，因此您应该能够手动设置它。我会尝试这个：hbase.metrics.showTableName，默认情况下应该是true，但我会尝试明确地将其设置为true。目前我正在研究HBase 0.90（Hortonworks HDP 1.1附带的那个），我甚至不必再使用SchemaMetrics.configureGlobally(conf)了。也许这是HBase 0.94中的一个bug（或者只是一种新方法）？

答案 1 :(得分：2)

我询问的相关工具已不再有效。 所以，我编写了一些Java代码，使用HBase API使用MapReduce进行批量加载。

Github链接：HBase Bulk Load

相关问题
最新问题