Question

我有大量的租赁商品数据集，我希望根据卧室数量生成每个城市的平均价格。我有以下类型的行： {( city: 'New York', num_bedrooms: 1, price: 1000.00 ), ( city: 'New York', num_bedrooms: 2, price: 2000.00 ), ( city: 'New York', num_bedrooms: 1, price: 2000.00 ), ( city: 'Chicago', num_bedrooms: 1, price: 4000.00 ), ( city: 'Chicago', num_bedrooms: 1, price: 1500.00 )}

使用Pig，我希望以下列格式获得结果： {( city: 'New York', 1: 1500.00, 2: 2000.00), ( city: 'Chicago', 1: 2750.00 )}

或者，我也可以处理这个问题： {( city: 'New York', num_bedrooms: 1, price: 1500.00), ( city: 'New York', num_bedrooms: 2, price: 2000.00), ( city: 'Chicago', num_bedrooms: 1, price: 2750.00 )}

我的计划是使用此数据创建条形图，其中包含沿X轴的卧室数量以及给定城市的Y轴上的价格。我已经能够按城市和卧室数量分组然后平均，但我不知道如何将数据放入我想要的格式。到目前为止，这就是我所拥有的： D = GROUP blah BY (city, num_bedrooms); C = FOREACH D GENERATE blah.city, blah.num_bedrooms, AVG(blah.price);

然而，这会导致城市和num_bedrooms在每次出现时重复出现！

Answer 1

输入：

New York,1,1000.00
New York,2,2000.00
New York,1,2000.00
Chicago,1,4000.00
Chicago,1,1500.00

方法1：

猪脚本：

rental_data = LOAD 'rental_data.csv'  USING  PigStorage(',') AS (city:chararray, num_bedrooms: long, price:double);
rental_data_grp_city = GROUP rental_data BY (city);
rental_kpi = FOREACH rental_data_grp_city {
    one_bed_room = FILTER rental_data BY num_bedrooms==1;
    two_bed_room = FILTER rental_data BY num_bedrooms==2;
    GENERATE group AS city, AVG(one_bed_room.price) AS one_bed_price, AVG(two_bed_room.price) AS tow_bed_price;
};

输出：DUMP rental_kpi：

 (Chicago,2750.0,)
 (New York,1500.0,2000.0)

方法2：

猪脚本：

rental_data = LOAD 'rental_data.csv'  USING  PigStorage(',') AS (city:chararray, num_bedrooms: long, price:double);
rental_data_grp_city = GROUP rental_data BY (city,num_bedrooms);
rental_kpi = FOREACH rental_data_grp_city {
    prices_bag = rental_data.price;
    GENERATE group.city AS city, group.num_bedrooms AS num_bedrooms, AVG(prices_bag) AS price; 
}

输出：DUMP rental_kpi：

(Chicago,1,2750.0)
(New York,2,2000.0)
(New York,1,1500.0)

如何在Pig Latin中生成大量数据的独特平均值？

1 个答案: