Question

我有两个数据集

Definition of schema A - Name, city, state
A= {
  Ram,    Sunnyvale,  CA
  Soju,   Austin,     TX
  Rathos, Bangalore,  Karnataka
  Mike,   Portland,   OR
}

B = {
    Ram,  Refund
    Soju, Refund
}

我想基于状态加入这两个表，并输出如下

Schema Definition - Name,City,State,RefundIssued (Yes/No)
  Ram,Sunnyvale,CA,yes
  Soju,Austin,TX,yes
  Rathos,Bangalore,Karnataka,no
  Mike,Portland, OR,no

我不确定如何指定我需要额外的列，哪些是逻辑

A = load 'data1.txt' using PigStorage(',') as (name: chararray,city: chararray,state: chararray);
B= load 'data2.txt' using PigStorage(',') as (name: chararray,type: chararray);
C = join A by name LEFT OUTER,B by name;  
D = foreach C generate A::name as firstname,B::type as charge_type;
--how to add new column which goes on refund issued as yes /no
store D into '1outdata.txt';

Answer 1

A = load 'data1.txt' using PigStorage(',') as (name: chararray,city: chararray,state: chararray);
B= load 'data2.txt' using PigStorage(',') as (name: chararray,type: chararray);
C = join A by name LEFT OUTER,B by name;  
D = foreach C generate A::name as name , A::city as city, A::state as state, (B::type == 'Refund' ? 'True' : 'False') as RefundIssued

请注意，由于bincond的工作原理，RefundIssues可以是'true'，'false'或null。如果要将null（左连接找不到匹配或字段值为null）转换为false，则使用：

E = foreach D generate name , city, state, (RefundIssued IS NULL ? 'False' : RefundIssued) as RefundIssued

加入Pig并创建一个新专栏

1 个答案: