Question

我正在使用Apache hadoop 1.1.1和Apache hbase 0.94.3。我想将数据从HDFS加载到HBASE。我写了猪脚本来达到目的。首先，我在habse中创建了hbase表，然后编写了pig脚本，将数据从HDFS加载到HBASE。但它没有将数据加载到hbase表中。不知道它在哪里。以下是用于创建hbase表的命令：

创建表'mydata'，'mycf'

以下是将数据从hdfs加载到hbase的猪脚本：

    A = LOAD '/user/hduser/Dataparse/goodrec1.txt' USING PigStorage(',') as (c1:int, c2:chararray,c3:chararray,c4:int,c5:chararray);   
    STORE A INTO 'hbase://mydata'
    USING org.apache.pig.backend.hadoop.hbase.HBaseStorage(
          'mycf:c1,mycf:c2,mycf:c3,mycf:c4,mycf:c5');

在说完剧本之后

2014-04-29 16:01:06,367 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 100% complete

2014-04-29 16:01:06,376 [main] ERROR org.apache.pig.tools.pigstats.PigStatsUtil - 1 map reduce job(s) failed!
2014-04-29 16:01:06,382 [main] INFO  org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics:

HadoopVersion   PigVersion      UserId  StartedAt       FinishedAt      Features
1.1.1   0.12.0  hduser  2014-04-29 15:58:07     2014-04-29 16:01:06     UNKNOWN

Failed!

Failed Jobs:
JobId   Alias   Feature Message Outputs
job_201403142119_0084   A       MAP_ONLY        Message: Job failed! Error -           JobCleanup Task Failure, Task: task_201403142119_0084_m_000001     hbase://mydata,

Input(s):
Failed to read data from "/user/hduser/Dataparse/goodrec1.txt"

Output(s):
Failed to produce result in "hbase://mydata"

Counters:
Total records written : 0
Total bytes written : 0
Spillable Memory Manager spill count : 0
Total bags proactively spilled: 0
Total records proactively spilled: 0

Job DAG:
job_201403142119_0084


2014-04-29 16:01:06,382 [main] INFO   org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Failed!

请在我出错的地方帮忙？

Answer 1

您在hbase的输出中指定了太多列。您有5个输入列和5个输出列，但HBaseStorage要求第一列是行键，因此输出中只应有4个

将数据从HDFS加载到HBASE

1 个答案: