Question

我的测试集是：

Onida|Lucid|18|Uttar Pradesh|232401|16200
Akai|Decent|16|Kerala|922401|12200
Lava|Attention|20|Assam|454601|24200
Zen|Super|14|Maharashtra|619082|9200
Samsung|Optima|14|Madhya Pradesh|132401|14200

我的映射器类：

public class UnitsSoldPerCompanyMapper extends Mapper<LongWritable,Text,Text,Text>{

    public void map(LongWritable inputKey, Text inputValue,Context context) throws IOException, InterruptedException{
        String[] lineArray= inputValue.toString().split("\\|");
        Text companyName = new Text(lineArray[0]);
        Text productName = new Text(lineArray[1]);
        context.write(companyName,productName);
    }
}

减速器类：

public class UnitsSoldPerCompanyReducer extends Reducer<Text,Iterable<Text>,Text,IntWritable>{

    public void reduce(Text companyKey,Iterable<Text> productName,Context context) throws IOException, InterruptedException{

        IntWritable counter1= new IntWritable();
        int counter =0;

        for(Text values : productName ){
            System.out.println(values);
            counter++;
        }
        counter1.set(counter);
        //IntWritable sum= new IntWritable(counter);
        context.write(companyKey, new IntWritable(1));
    }
}

驱动程序类：

public class UnitsSoldPerCompanyDriver {

public static void main(String[] args) throws IOException, ClassNotFoundException, InterruptedException {

    Configuration conf = new Configuration();// To set job related
                                                // configuration

    // @SuppressWarnings("deprecation")
    @SuppressWarnings("deprecation")
    Job job = new Job(conf, "TaskofJob");
    job.setJarByClass(UnitsSoldPerCompanyDriver.class);

    // Job job = new Job(conf,"TvSalesAcrossLocations");

    FileInputFormat.addInputPath(job, new Path(args[0]));
    FileOutputFormat.setOutputPath(job, new Path(args[1]));

    job.setMapperClass(UnitsSoldPerCompanyMapper.class);
    job.setReducerClass(UnitsSoldPerCompanyReducer.class);

    BasicConfigurator.configure();
    job.setMapOutputKeyClass(Text.class);
    job.setMapOutputValueClass(Text.class);

    job.setOutputKeyClass(Text.class);
    job.setOutputValueClass(IntWritable.class);

    // job.setInputFormatClass(TextInputFormat.class);
    // job.setOutputFormatClass(TextOutputFormat.class);

    job.waitForCompletion(true);

}

我出去了：

Akai    Decent
Lava    Attention
Lava    Attention
Lava    Attention
NA  Lucid
Onida   NA
Onida   Decent
Onida   Lucid
Onida   Lucid
Samsung Super
Samsung Super
Samsung Super
Samsung Decent
Samsung Optima
Samsung Optima
Samsung Optima

然而，我试图找到每家公司销售的单位。

Answer 1

我相信输出是由身份（默认）reducer（它只输出映射器键和带有制表符的值）而不是你的。不知道为什么会这样，但我怀疑是BasicConfigurator.configure（）;

在ResourceManager UI中，您可以验证mapred.reducer.class，转到作业，在左侧菜单中，您可以看到使用的实际作业属性。

Answer 2

我错误地扩展了Reducer，Text，IntWritable＆gt;，这应该是Reducer

由于我没有在reduce方法中使用@Override注释，所以在编译时没有指出。

当我指定IntWritable时，为什么我的map reduce程序会在Text中输出

2 个答案: