在Pig脚本中过滤列

Question

我正在从CSV加载Pig中的数据。 加载数据后，我需要过滤掉列。

exportAllProductsCleaned = FOREACH exportAllProducts 
generate $0, $1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18, $19, $20, $26, $27, $28, $29, $30, $31, $32, $33

有没有一种方法可以只指定

1. 我需要删除的列 OR
2. 我需要的列数范围。 $ 1-15然后$ 18 - $ 30

有可能吗？

Answer 1

是的，您可以使用＆＃39; ..＆＃39; convention.Refer Support project range expression

exportAllProductsCleaned = FOREACH exportAllProducts GENERATE $0, $1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18, $19, $20, $26, $27, $28, $29, $30, $31, $32, $33;
exportAllProductsFiltered = FOREACH exportAllProductsCleaned GENERATE $1 .. $15,$18 .. $30;