在map-reduce

时间:2016-08-28 15:56:22

标签: java hadoop static mapreduce

我有一个mapper和reducer以及一个util类。

- The util class has a static class variable int a=0 
  and a static method doSomething() which calls another static method 
  add() which increment a by 1.
- I call the UtilClass.doSomething() from the reducer.
- When I try to print the final value of "a" in 
  doSomething() using 
  context.getCounter("doSomethign method","value of a").increment(1);,
  the value says 1229 which is weird since the reduce task received only 240 keys
- A total of 30 reducers are spawned. 

以下是示例代码块。我正在避免一些样板代码。 注意:这是以分布式方式运行的,而不是独立的。

class MyMapper extends Reducer{
 public reduce(<params>){
   UtilClass.doSomething(context);
 }
}

class UtilClass{
  public static int a=0;
  public static void add(){
    a=a+1;
  }
  public static void doSomething(Reducer.Context context){
    add();
    // This is printing 1229 in the end, instead of 240 which is the number of key given to the reducer
    context.getCounter("doSomethign method","value of a").increment(1);
  }
}

这引出了以下问题

  1. 我读到每个Map / reduce任务都在不同的JVM上运行。但如果是这样,为什么变量得到这个奇怪的1229值?每个reduce任务应该只为每个键调用doSomething()一次,“a”的最终值应该是240.
  2. 如果我尝试创建一个util类的对象并调用Object.doSomething()(使方法非静态但保持“静态”),那么我得到240。

    1. 我在静态基础中遗漏了什么?

0 个答案:

没有答案