Question

Pig Latin的初学者。我试图计算输入文件中多个字符串的出现次数。

现在搜索必须不区分大小写。我知道猪有一个LOWER内置函数，但我该如何使用呢？

例如（输入文件）：

28-Oct-13,7:00PM,Viraj,New to hadoop ! Eager to learn.
31-Dec-14,3:00PM,‏Vanguard,Designers, Developers, Doers, don't miss this upcoming San Francisco Hadoop

我需要像hadoop, dec, learn, python

这样的字符串数

hadoop 2
dec 1
learn 1
python 0

如何在猪拉丁语中进行搜索？

谢谢。

Answer 1

你能试试吗？

<强>输入

28-Oct-13,7:00PM,Viraj,New to hadoop ! Eager to learn.
31-Dec-14,3:00PM,?Vanguard,Designers, Developers, Doers, don't miss this upcoming San Francisco Hadoop

<强> PigScript：

A = LOAD 'input' AS (line:chararray);
B = FOREACH A GENERATE  FLATTEN(TOKENIZE(LOWER(line))) as word;
C = FOREACH B GENERATE ((word matches '.*hadoop.*'? 1:0)) as t1,((word matches '.*dec.*'?1:0)) as t2,((word matches '.*learn.*'?1:0)) as t3,((word matches '.*python.*'?1:0)) as t4;
D = GROUP C ALL;
E  = FOREACH D GENERATE FLATTEN(TOBAG(CONCAT('hadoop',' ',(chararray)SUM(C.t1)),CONCAT('dec',' ',(chararray)SUM(C.t2)),CONCAT('learn',' ',(chararray)SUM(C.t3)),CONCAT('python',' ',(chararray)SUM(C.t4))));
DUMP E;

<强>输出：

(hadoop 2)
(dec 1)
(learn 1)
(python 0)

Pig Latin中不区分大小写的搜索

1 个答案: