在FOREACH中计算SUM

时间:2012-11-14 18:50:59

标签: hadoop mapreduce apache-pig

假设我有以下

DATA = foreach INPUT {
   //..
   generate group, count(name) as total;
}

我最终会得到一个关键,其中键按名称分组

('mike', 'someprop', 10)
('mike', 'otherprop', 3)
('doug', 'xprop', 5)
...

我希望得到每个名字前10名的总和

ALIAS = group DATA by name;
RESULT = foreach ALIAS {
   SORTED = ORDER DATA by total desc;
   TOP10 = LIMIT SORTED 10;

   //doesn't work! can't have GROUP inside FOREACH
   AGG = group TOP10 ALL;
   TOPTOTAL = foreach AGG generate SUM(AGG.total);

   generate group, TOPTOTAL;
}

如何计算foreach关系的值(SUM,COUNT,ETC)?目前,无法在foreach中应用GROUP ALL

1 个答案:

答案 0 :(得分:0)

SUM只是一个以包作为参数的函数,您可以通过从TOP10投影创建此包:

ALIAS = group DATA by name;
RESULT = foreach ALIAS {
   SORTED = ORDER DATA by total desc;
   TOP10 = LIMIT SORTED 10;
   generate group, SUM(TOP10.total);
}