Question

我有一个mapper

public class BuildGraph{
   public void config(JobConf job){ <==this block doesnt seems to be exexcuting at all :(
    super.configure(job);
    this.currentId = job.getInt("currentId",0);
    if (this.currentId!=0){ 
    // I call a method from differnt class to build a distributed cache
    }
   }
  public void map(....){
....  
}

}

现在是调用它的主要代码..

public void run( String params,curId){
 JobConf conf = new JobConf(classname.class);
 conf.setInt("currentId",299); <--note this i am setting the value here
 conf.setMapperClass(BuildGraph.class);
 //.... 
  JobClient.runJob(conf);
}

但问题是代码中的config方法没有执行，好像“currentId”在主循环中返回299但在mapper类中根本没有设置。我做错了什么

链接到完整代码 http://pastebin.com/DRPXUm62

Answer 1

您似乎没有使用正确的合同，因为您没有展开MapReduceBase而没有实施Mapper。此方法也应该被称为configure而不是config。尝试这样的事情：

public class BuildGraph extends MapReduceBase implements Mapper<K, V, K, V> {
    public void configure(JobConf job) {
        // this will get called once at the beginning of the task
    }

    public void map(...) {
        ...
    }
}

新手hadoop ..分布式缓存

1 个答案: