Question

我正在尝试在文本文件中查找特定字符串并计算该特定字符串，并且我编写了以下代码，现在我不确定我应该在这里更改，请回复：

错误：16/04/29 04:30:27 INFO mapreduce.Job：工作   job_1461630807194_0010在uber上运行   模式：false 16/04/29 04:30:27 INFO mapreduce.Job：地图0％减少0％   16/04/29 04:34:08 INFO mapreduce.Job：地图100％减少0％16/04/29   04:34:15 INFO mapreduce.Job：任务ID：   attempt_1461630807194_0010_m_000
  000_0，状态：FAILED错误：java.io.IOException：类型不匹配   来自地图的密钥：期望的org.apache.h
  adoop.io.Text，收到org.apache.hadoop.io.LongWritable           at org.apache.hadoop.mapred.MapTask $ MapOutputBuffer.collect（MapTask.java   ：1072）

以下是代码：

我的Mapper类：

package Receipt;
import java.io.IOException;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Mapper;

public class ReceiptMapper extends Mapper<LongWritable,Text,Text,Text>
{

    public static class Stringmap extends Mapper<LongWritable, Text, Text, IntWritable> {
        public void map(LongWritable key, Text value, Context context)
                throws IOException, InterruptedException {
            String str = value.toString();

            String findStr = "not created";
            int lastIndex = 0;
            int count = 0;
            while (lastIndex != -1) {

                lastIndex = str.indexOf(findStr, lastIndex);

                if (lastIndex != -1) {
                    count++;
                    lastIndex += findStr.length();

                }
            }
            if (str.contains(findStr)) {
                context.write(new Text(findStr), new IntWritable(count));
            }

        }
    }
}





public class ReceiptReducer extends Reducer<Text, IntWritable, Text, IntWritable> 
{
    public void reduce(Text key, Iterable<IntWritable> values,Context context) throws IOException, InterruptedException {
        int sum = 0;
        for (IntWritable val : values) 
        {
            sum += val.get();
        }    
        context.write(key, new IntWritable(sum));
    }
}





public class ReceiptCount {
    @SuppressWarnings("deprecation")
    public static void main(String[] args)  throws IOException, InterruptedException, ClassNotFoundException{ {
        // TODO Auto-generated method stub
        Configuration conf = new Configuration();
        Job job = new Job(conf, "Stringfound");

        job.setJarByClass(ReceiptCount.class);

        job.setMapperClass(ReceiptMapper.class);
        job.setReducerClass(ReceiptReducer.class);
        job.setMapOutputKeyClass(LongWritable.class);

        job.setMapOutputValueClass(LongWritable.class);
        job.setOutputKeyClass(Text.class);
        job.setOutputValueClass(IntWritable.class);

        FileInputFormat.addInputPath(job, new Path(args[0]));
        FileOutputFormat.setOutputPath(job, new Path(args[1]));
        //setMapOutputKeyClass();

        //setMapOutputValueClass();
        Path outputDir = new Path( args[1] );
        outputDir.getFileSystem( conf ).delete( outputDir, true );
        FileOutputFormat.setOutputPath( job, outputDir );

        System.exit(job.waitForCompletion(true) ? 0 : 1);
    }


    }
}

Answer 1

，当你开始

public class ReceiptMapper extends Mapper<LongWritable,Text ....

你正在做一个

context.write(new Text(findStr), new IntWritable(count));

long不是字符串，因此映射器失败。

Map中的问题减少IOException

1 个答案: