Question

我写了一个猪脚本如下：

REGISTER /home/hduser/hbase/lib/zookeeper-*.jar
REGISTER /home/hduser/hbase/lib/hbase-*.jar
REGISTER /home/hduser/hbase/lib/hadoop*.jar
REGISTER /home/hduser/pig/lib/hbase-0.94.1.jar
REGISTER /home/hduser/pig/lib/zookeeper-3.4.5.jar
REGISTER /home/hduser/pig/lib/piggybank.jar


STOCK_2008 = LOAD 'hdfs:/user/file.txt' using PigStorage(',') AS (no:int, name:chararray, digit:int);
DUMP STOCK_2008 ;
STORE STOCK_2008 INTO 'hbase:/hi' USING org.apache.pig.backend.hadoop.hbase.HBaseStorage('hi_data:name, hi_data:digit');

我有猪版本0.13.0 和 hbase版本0.98.8 和 hadoop 2.5.1。

我正面临问题：

2014-12-25 15:54:54,028 [main] INFO  org.apache.pig.newplan.logical.optimizer.LogicalPlanOptimizer - {RULES_ENABLED=[AddForEach, ColumnMapKeyPrune, GroupByConstParallelSetter, LimitOptimizer, LoadTypeCastInserter, MergeFilter, MergeForEach, PartitionFilterOptimizer, PushDownForEachFlatten, PushUpFilter, SplitFilter, StreamTypeCastInserter], RULES_DISABLED=[FilterLogicExpressionSimplifier]}
2014-12-25 15:54:56,637 [main] INFO  org.apache.pig.data.SchemaTupleBackend - Key [pig.schematuple] was not set... will not generate code.
2014-12-25 15:54:56,744 [main] INFO  org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated. Instead, use fs.defaultFS
2014-12-25 15:54:56,753 [main] INFO  org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 1
2014-12-25 15:54:56,757 [main] INFO  org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths to process : 1
2014-12-25 15:54:57,585 [main] INFO  org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter - Saved output of task 'attempt__0001_m_000001_1' to hdfs://masternode/tmp/temp-1402587382/tmp-1688408384/_temporary/0/task__0001_m_000001
2014-12-25 15:54:57,720 [main] INFO  org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated. Instead, use fs.defaultFS
2014-12-25 15:54:57,725 [main] WARN  org.apache.pig.data.SchemaTupleBackend - SchemaTupleBackend has already been initialized
2014-12-25 15:54:57,731 [main] INFO  org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 1
2014-12-25 15:54:57,731 [main] INFO  org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths to process : 1
(1,one,1)
(2,two,2)
(3,three,3)
2014-12-25 15:54:57,802 [main] INFO  org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated. Instead, use fs.defaultFS

2014-12-25 15:54:57,902 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2998: Unhandled internal error. org.apache.hadoop.hbase.util.Bytes.equals([BLjava/nio/ByteBuffer;)Z

Details at logfile: /home/hduser/pig_1419503089087.log

Answer 1

删除所有注册声明并使用＆＃39; hbase：// hi＆＃39;而不是＆＃39; hbase：/ hi＆＃39;

因为您正在使用hbase 0.98.8而您正在导入hbase-0.94.1.jar

尝试使用猪脚本面临问题将数据加载到hbase

1 个答案: