Question

Input:
Visit ID             Events
101                154,2,135
124                1, 120, 1050,2302
139                200, 150, 1, 320
140                30023, 200

新手猪。想知道如何使用Pig脚本列出事件中包含“1”的visitID行。

谢谢！

我试过的代码：

a = LOAD '/user/a6000518-a/AdobeHourlySampleHit/hit_data.tsv' using PigStorage('\t');  
b= foreach a GENERATE REGEX_EXTRACT_ALL($2, '(.*,1,.*|1,.*|.*,1)') as post_event_list;
c= FILTER b BY $0 is not NULL;
d= DISTINCT c;
dump d;

这仅打印行包含“1”的事件列。如果我使用visitID生成，我得到的结果不正确。我想打印visitID以及包含“1”的事件。

Answer 1

你可以写一个python udf并查看字符串中是否存在有问题的字符;可能会让事情变得更简单。

python udf ：

#!/usr/bin/python

@outputSchema("flg:int")
def tuple_contains(tup, val):
    try:
        if val in tup:
            return 1
        else:
            return 0
    except:
        return 0

<强>脚本：

REGISTER /path/to/jars/tuple_contains.py USING jython AS udf;

data = LOAD 'data' AS (visit_id:chararray, event_list:chararray);
A = FILTER data BY udf.tuple_contains(STRSPLIT(event_list, ','), '1') == 1;
B = FOREACH A GENERATE visit_id, event_list;  -- ... other columns
DUMP B;

<强>输出：

124    1,120,1050,2302
139    200,150,1,320

Answer 2

我明白了。可能不是有效的编码方式，但可以准确地获得输出。

a = LOAD '/user/a6000518-a/AdobeHourlySampleHit/hit_data.tsv' using PigStorage('\t');  
b= FILTER a BY $283 == '0';  
c= FILTER b BY $298 != '5' AND $298 != '8' AND $298 != '7' AND $298 != '9';
d= FOREACH c GENERATE CONCAT(CONCAT(CONCAT($780,$781),$942),$948) as (visitID:bytearray), $614 as post_event_list, $656 as post_product_list;
e= FILTER d BY post_event_list != ' ' OR post_event_list != '' OR post_event_list is not NULL;
f= FOREACH e GENERATE REGEX_EXTRACT_ALL(post_event_list, '(.*,1,.*|1,.*|.*,1)') as purchase_event, visitID, post_product_list;
g= FILTER f BY $0 is not NULL;
h= dump g;

如何列出包含特定字符串的元组？

2 个答案: