Question

在为自定义InputFormat

设置作业时，我遇到了一些错误

以下是我的代码

package com.nline_delimiter;

import java.io.IOException;

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.mapreduce.Job;
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;



public class NL_driver {

public static void main(String [] args) throws IOException, InterruptedException, ClassNotFoundException
{
    Configuration conf=new Configuration(true);

    Job job_run =new Job(conf);

    job_run.setJobName("nline input format each line seperate wth delimiter");

    job_run.setJarByClass(NL_driver.class);

    job_run.setMapperClass(NL_mapper.class);
    job_run.setReducerClass(NL_reducer.class);
    job_run.setInputFormatClass(NL_inputformatter.class);;


    job_run.setMapOutputKeyClass(Text.class);
    job_run.setMapOutputValueClass(IntWritable.class);
    job_run.setOutputKeyClass(Text.class);
    job_run.setOutputValueClass(IntWritable.class);


    FileInputFormat.setInputPaths(job_run,new Path("/home/hduser/input_formatter_usage.txt"));
    FileOutputFormat.setOutputPath(job_run, new Path("/home/hduser/input_formatter_usage"));

    job_run.waitForCompletion(true);
}
}

The Line

job_run.setInputFormatClass(NL_inputformatter.class)

显示错误

NL_inputformatter是一个自定义的Inputformatter类，它扩展了FileInputFormat

我是否需要为setInputFormatClass导入一些内容，因为Eclipse中的默认错误检查要求我将setInputFormatClass更改为setOutFormatClass但不要求任何导入。

NL_inputformatter的源代码如下。

package com.nline_delimiter;

import java.io.IOException;

import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapred.FileInputFormat;
import org.apache.hadoop.mapred.FileSplit;
import org.apache.hadoop.mapred.InputSplit;
import org.apache.hadoop.mapred.JobConf;
import org.apache.hadoop.mapred.RecordReader;
import org.apache.hadoop.mapred.Reporter;

public class NL_inputformatter extends FileInputFormat<Text, IntWritable>{

@Override
public RecordReader<Text, IntWritable> getRecordReader(InputSplit input,
        JobConf job_run, Reporter reporter) throws IOException {
    // TODO Auto-generated method stub
    System.out.println("I am Inside the NL_inputformatter class");
    reporter.setStatus(input.toString());
    return new NL_record_reader(job_run, (FileSplit)input);


}

}

我们将不胜感激。

Answer 1

这是因为您使用旧FileInputFormat API中的Hadoop和新版import org.apache.hadoop.mapred.FileInputFormat;。您必须更改导入和实施：

import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;

到

inputSlider

设置job.setInputFormatClass时出错

1 个答案: