MR中有什么问题?

时间:2017-01-06 02:44:34

标签: java mapreduce hdfs cloudera-cdh

我在CDH集群中运行一个MR程序,最后显示成功和输出文件成功,但开头有很多错误:

enter image description here

enter image description here

这是我的代码:

    Configuration hdfsConf = new Configuration();
    System.setProperty("HADOOP_USER_NAME", "hdfs");
    hdfsConf.set("HADOOP_USER_NAME", "hdfs");
    String jobName = "AIAD-MR";
    Job job = Job.getInstance(hdfsConf, jobName);
    Path path = new Path(inputDirPath);
    Path path1 = new Path(outPath);
    FileSystem fileSystem = path.getFileSystem(hdfsConf);
    FileSystem fileSystem1 = path1.getFileSystem(hdfsConf);
    if (fileSystem.exists(path)) {
        FileInputFormat.setInputDirRecursive(job, true);
        FileInputFormat.addInputPath(job, path);
    }
    if (fileSystem1.exists(path1)) {
        fileSystem1.delete(path1, true);
    }
    FileOutputFormat.setOutputPath(job, new Path(outPath));

0 个答案:

没有答案