使用PIG拉丁将数组转置为行

时间:2018-06-30 06:17:51

标签: apache-pig

如何将BAG中的ARRY元素转换为多行,例如:

我的输入:

元组,ARRAY_ELEM (32,{(1,emp,3271409712),(2,emp,3271409712)})

输出 (32,1,emp,3271409712) (32,2,emp,3271409712)

1 个答案:

答案 0 :(得分:0)

您可能需要两次拨打FLATTEN。 请注意,元组上的FLATTEN只会将元组中的每个字段提升为顶级字段。 手提袋上的FLATTEN会产生手提袋中每个记录与GENERATE中所有其他表达的叉积。

A = load 'test.txt' using PigStorage() as (a0:int, t1:(a1:int, b1 {(a3:int,a4:chararray,a5:chararray)}));
describe A;
B = FOREACH A GENERATE FLATTEN(t1);  
describe B;
C = FOREACH B GENERATE a1, FLATTEN(b1);
describe C;
dump C;

输出

A: {a0: int,t1: (a1: int,b1: {(a3: int,a4: chararray,a5: chararray)})}
B: {t1::a1: int,t1::b1: {(a3: int,a4: chararray,a5: chararray)}}
C: {t1::a1: int,t1::b1::a3: int,t1::b1::a4: chararray,t1::b1::a5: chararray}
(32,1,emp,3271409712)
(32,2,emp,3271409712)