Question

我被困在这几个小时，我无法弄清楚我做错了什么。我有一个关系＆＃34;分组＆＃34;使用

的架构

    grouped: {seedword: chararray,baggy: {outertup: (groupy: (seedword: chararray,coword: chararray))}}

关系的样本是：（自动，{（（自动，汽车）），（（轿车，卡车））}）

我需要只生成种子词和一群牛郎。在我的例子中，我想要

（汽车，（汽车，卡车））。

我试过了：

 FOREACH grouped GENERATE baggy::outertup.groupy.coword;

 FOREACH grouped GENERATE baggy.outertup.groupy.coword;
 FOREACH grouped GENERATE baggy.groupy.coword;

并且这些都不起作用，并且给我错误消息说没有这样的字段。请帮忙！ !!

我的更多代码：

keywords = LOAD 'merged' USING as ( seedword:chararray, doc:chararray);

---COUNT HOW MANY DOCUMENTS EACH WORD IS IN
group_by_seedword = GROUP keywords BY $0;

invert_index = FOREACH group_by_seedword GENERATE $0 as seedword:chararray, keywords.$1;
word_doc_count= FOREACH invert_index GENERATE seedword, COUNT($1);

-- map words to document
words_in_doc= GROUP keywords BY doc;
word_docs = FOREACH words_in_doc GENERATE group AS doc, keywords.seedword;
--(document:(keyword, keyword, keyword...))

--map words to their cowords in doc
temp_join = JOIN keywords BY doc,word_docs BY doc;
--DUMP temp_join;
cowords_by_doc = FOREACH temp_join GENERATE $0 as seedword:chararray, $3 as cowords;

cowords_interm=  FOREACH cowords_by_doc GENERATE seedword, FLATTEN(cowords);
cowords = FILTER cowords_interm BY (seedword!=$1);---GETS RID OF SINGLE DOC WORD; 
temp_join_count1 = JOIN cowords BY $0, word_doc_count BY seedword; 

-- GETS WORDS THAT OCCURE BY THEMSELVES IN A SINGLE DOCUMENT
G = JOIN cowords_interm BY $0 LEFT OUTER, cowords by $0;
orph_word = FILTER G BY $2 is null;
orph_word_count = FOREACH orph_word GENERATE $0,null, 0;

temp_join_count= UNION temp_join_count1, orph_word_count; 

inter_frac = FOREACH temp_join_count GENERATE $0 as seedword:chararray, $1 as coword:chararray, 1.0/$3 as frac:double;
inter_frac_combine = GROUP inter_frac BY (seedword, coword); 
inter_frac_sum = FOREACH inter_frac_combine GENERATE $0 , SUM(inter_frac.frac) as frac:double;

filtered = FILTER inter_frac_sum BY ($1 >=$relatedness_ratio);
grouped= GROUP filtered by $0.seedword;
g = FOREACH grouped GENERATE group as seedword:chararray, filtered.$0;   
named = FOREACH g GENERATE $0 as seedword:chararray, $1 as baggy:bag{(outertup:tuple(groupy:tuple(seedword:chararray, coword:chararray)))};

您可以尝试的输入文件应该是这样的：

car doc1.txt
auto doc1.txt
bunny doc2.txt
ball doc2.txt
toy car doc2.txt
random doc3.txt

plane doc3.txt

Answer 1

我有一个类似的问题，我无法引用内部元组。我的解决方案是压缩数据，然后进行一些过滤和分组。干杯 V

在PIG中引用嵌套元组中的字段;

1 个答案: