猪的未命名列的总和

时间:2016-07-08 11:49:47

标签: hadoop apache-pig hcatalog bigdata

 shipnode,delivery_method ,<unnamed>
 (9935,PICK,2)
 (9960,PICK,2)
 (9969,PICK,1)
 (9963,SHP,1)
 (9989,SHP,1)
 (9995,SHP,1)
 (9965,SHP,1)
 (9995,SHP,1)

这是

的输出
 grunt> group_all_shipnode = GROUP
 >> union_all 
 >> BY(
 >> shipnode,delivery_method
 >> )
 >> ;

最后一列未命名,现在我要生成 作为shipnode和delivery_node的分组,并将第三列的总和作为

 (9935,PICK,2)
 (9960,PICK,2)
 (9969,PICK,1)
 (9963,SHP,1)
 (9989,SHP,1)
 (9995,SHP,2) <<-------      sum of similar 
 (9965,SHP,1)

我正在尝试这个:

 grunt> sum_group_all_shipnode =FOREACH group_all_shipnode 
 >> GENERATE FLATTEN(group) as(shipnode:chararray, delivery_method:chararray),
 >> sum($1.$2);

产生错误:

ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1070: Could not resolve sum using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]

1 个答案:

答案 0 :(得分:0)

而不是$ 1. $ 2,它必须是你的load语句中的关系。 例如,假设您要将数据加载到关系A。

A = LOAD 'data.csv' USING PigStorage(',');
group_all_shipnode = GROUP A BY ($1,$2);
sum_group_all_shipnode = FOREACH group_all_shipnode 
                         GENERATE 
                             FLATTEN(group) AS (shipnode:chararray, delivery_method:chararray),
                             SUM(A.$2);