Question

我有一个包含用户的CSV文件（tweetid，tweets，userid）。

396124436476092416,"Think about the life you livin but don't think so hard it hurts Life is truly a gift, but at the same it is a curse",Obey_Jony09
396124436740317184,"“@BleacherReport: Halloween has given us this amazing Derrick Rose photo (via @amandakaschube, @ScottStrazzante) http://t.co/tM0wEugZR1” yes",Colten_stamkos
396124436845178880,"When's 12.4k gonna roll around",Matty_T_03

现在我需要编写一个Pig查询，它返回包含“最喜欢”一词的所有推文，按推文ID排序。

为此，我有以下代码： A = load '/user/pig/tweets' as (line); B = FOREACH A GENERATE FLATTEN(REGEX_EXTRACT_ALL(line,'(.*)[,”:-](.*)[“,:-](.*)')) AS (tweetid:long,msg:chararray,userid:chararray); C = filter B by msg matches '.*favorite.*'; D = order C by tweetid;

正则表达式如何在以期望的方式拆分输出？

我尝试使用REGEX_EXTRACT而不是REGEX_EXTRACT_ALL，因为我发现更简单，但除了仅提取推文外无法使代码工作：

B = FOREACH A GENERATE FLATTEN(REGEX_EXTRACT(line,'[,”:-](.*)[“,:-]',1)) AS (msg:chararray);

上面的别名给我发了推文，但如果我使用REGEX_EXTRACT来获取tweet_id，我就得不到想要的o / p：B = FOREACH A GENERATE FLATTEN(REGEX_EXTRACT(line,'(.*)[,”:-]',1)) AS (tweetid:long);

(396124554353197056,"Just saw @samantha0wen and @DakotaFears at the drake concert #waddup")
(396124554172432384,"@Yutika_Diwadkar I'm just so bright ")

(396124554609033216,"@TB23GMODE i don't know, i'm just saying, why you in GA though? that's where you from?")

(396124554805776385,"@MichaelThe_Lion me too ")

(396124552540852226,"Happy Halloween from us 2 @maddow &amp; @Rev_AlSharpton :) http://t.co/uC35lDFQYn")
grunt>

请帮忙。

Answer 1

无法发表评论，但从查看并测试出来，看起来你的正则表达式中的引号与csv中的引号不同。

csv中的

"

正则表达式代码中的

”。

要获得tweetid试试这个：

B = FOREACH A GENERATE FLATTEN(REGEX_EXTRACT(line,'.*(,")',1))  AS (tweetid:long);

在hadoop中使用猪的正则表达式

1 个答案: