Hadoop 2.6.4 - 使用Tool界面时的异常

时间:2016-04-25 17:06:20

标签: java hadoop nullpointerexception

我在Java中使用Hadoop编写简单代码。当我尝试将其运行到jar文件并从CLI运行时 - 它正在运行。但是,当我使用Tool interface nad ToolRunner进行操作时,我有一个例外:

public class Main extends Configured implements Tool {

    //mvn clean package antrun:run@deploy
    public static void main(String[] args) throws Exception {
        int res = ToolRunner.run(new Configuration(), new Main(), args);
        System.exit(res);
    }

    @Override
    public int run(String[] args) throws Exception {
        Configuration conf = this.getConf();
        conf.set("mapred.job.tracker", "hadoop-master:8021");
        conf.set("fs.default.name", "hdfs://hadoop-master:9000/user/vagrant");
        conf.set("hadoop.job.ugi", "vagrant");
        System.setProperty("HADOOP_USER_NAME", "vagrant");
        FileSystem fs = FileSystem.get(conf);
        System.out.println(fs.isDirectory(new Path("/user/vagrant")));
        //fs.mkdirs(new Path("input"));

        Job job = Job.getInstance(conf, "passent");
        job.setJarByClass(Main.class);
        job.setMapperClass(PasswordMapper.class);
        job.setReducerClass(EntrophyPassReducer.class);
        job.setSortComparatorClass(EntrophyDescComparator.class);
        job.setOutputKeyClass(DoubleWritable.class);
        job.setOutputValueClass(Text.class);
        FileInputFormat.addInputPath(job, new Path("input"));
        FileOutputFormat.setOutputPath(job, new Path("output"));
        return job.waitForCompletion(true) ? 0 : 1;
    }

}

这是我的代码(仅限配置):

r = a + (b-a).*rand(N,1)

有人能帮助我吗? :) 提前谢谢!

0 个答案:

没有答案