Question

我在txt文件(Format: Firstname,Lastname,age,sex)中有这个smaple数据集：

(Eric,Ack,27,M)
(Jenny,Dicken,27,F)
(Angs,Dicken,28,M)
(Mahima,Mohanty,29,F)

我想要显示年龄大于27岁的员工的age和firstname。我在进行了大量工作并寻找指示后陷入困境：

我正在使用以下方法加载此数据集：

tuple_record = LOAD '~/Documents/Pig_Tuple.txt' AS (details:tuple(firstname:chararray,lastname:chararray,age:int,sex:chararray));

描述给我这种格式：

describe tuple_record
tuple_record: {details: (firstname: chararray,lastname: chararray,age: int,sex: chararray)}

然后我用这个来压扁记录：

flatten_tuple_record = FOREACH tuple_record GENERATE FLATTEN(details);

描述扁平化给了我这个：

describe flatten_tuple_record
flatten_tuple_record: {details::firstname: chararray,details::lastname: chararray,details::age: int,details::sex: chararray}

现在我想根据年龄过滤这个：

filter_by_age = FILTER flatten_tuple_record BY age > 27;

然后我根据年龄做了一个小组：

group_by_age = GROUP filter_by_age BY age;

现在显示名字和年龄;我尝试了这个，但它没有奏效：

display_details = FOREACH group_by_age GENERATE group,firstname;

以下是错误消息：

2015-02-01 08:39:37,752 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1025: 
<line 5, column 54> Invalid field projection. Projected field [firstname] does not exist in schema: group:int,filter_by_age:bag{:tuple(details::firstname:chararray,details::lastname:chararray,details::age:int,details::sex:chararray)}

请指导。

Answer 1

您的猪的陈述看起来不错，但按年龄过滤数据后，您可以直接获得名字和年龄。请遵循以下声明：

tuple_record = LOAD '/user/cloudera/Pig_Tuple.txt' AS (details:tuple(firstname:chararray,lastname:chararray,age:int,sex:chararray));

describe tuple_record;

flatten_tuple_record = FOREACH tuple_record GENERATE FLATTEN(details);

describe flatten_tuple_record;

filter_by_age = FILTER flatten_tuple_record BY age > 27;

details = FOREACH filter_by_age GENERATE firstname, age;

dump details;

更新

我们甚至可以跳过FLATTEN声明：

tuple_record = LOAD '/user/cloudera/Pig_Tuple.txt' AS (details:tuple(firstname:chararray,lastname:chararray,age:int,sex:chararray));

describe tuple_record;

filter_by_age = FILTER tuple_record BY details.age > 27;

details = FOREACH filter_by_age GENERATE details.firstname, details.age;

dump details;

在这两种情况下，结果将是：

(Angs,28)
(Mahima,29)

无法使用Pig FOREACH显示数据

1 个答案:

更新