Question

我有两个名为A.txt和B.txt的文件。

A.txt:                    B.txt
id  name    des          id  name
1   one     0            1   apple   
2   two     1            2   pine  
3   three   0            3   orange

在这里，我在两个文件中都有相同的字段“id”。我使用了两个mapper类。一个用于A.txt，另一个用于B.txt。有一个减速器类。

In MapperA.class "des" is checked and if "0" then its id is sent as key and value as name. 
In MapperB.class id and name is sent as key and value.
Now, in reducer I need to determine the "id" from MapperA and check with "id" from MapperB and if present, then send key as "id" and value as "name". 
How can we compare the key and value from two mapper in a reducer to get desired result?

reducer的所需输出是：

id      name
1       apple
3       orange

Answer 1

你不需要在reduce函数中进行比较过程。每个输入A.txt和B.txt

将有2个单独的映射器类

MAPPER FOR A.txt if（des == 0）然后写id，值 - ＆gt; des＆gt; // des将是TextWritable（“0”）其他不写任何东西

MAPPER FOR B.txt

id，value - ＆gt; apple或orange或pine ...＆gt;

<强>减速

输入将是＆lt; 1，{0，apple}＆gt; ＆lt; 2，{pine}＆gt; ＆lt; 3，{0，orange}＆gt;

然后在减少过程中

Boolean shouldBeWritten=false;
foreach value
{
  char firstLetter = first char of value;
  if( firstletter == 0 )
   shouldBeWritten=true;
}
if(shouldBeWritten)
write(key, value);

在这个实现中你必须获得fruitname的第一个字母，这个字母效率不高，但你不必在第一个映射器中发送不必要的对。

这只是一个由你决定的例子。

决定还取决于a.txt和b.txt的大小。

如何使用两个映射器到一个reducer的键和值？

1 个答案: