Question

请寻求帮助。执行mapreduce作业，但不产生任何输出。这是一个简单的程序，可以计算文件中单词的总数。我开始非常简单地确保它可以与包含以下内容的一行的txt文件一起使用：

小国第二大国第二小食品出口国第二秒秒

不幸的是，关于下一个看点的任何建议将不胜感激。我已经剪切并粘贴了输出日志的最后一位。

{% if user.username in staff_users %}
   <li class="nav-item pl-3">
     <a class="btn btn-primary" role="button" href="/productadmin">Admin</a>
   </li>
{% endif %}

File System Counters
    FILE: Number of bytes read=890
    FILE: Number of bytes written=947710
    FILE: Number of read operations=0
    FILE: Number of large read operations=0
    FILE: Number of write operations=0
Map-Reduce Framework
    Map input records=1
    Map output records=1
    Map output bytes=87
    Map output materialized bytes=95
    Input split bytes=198
    Combine input records=0
    Combine output records=0
    Reduce input groups=1
    Reduce shuffle bytes=95
    Reduce input records=1
    Reduce output records=1
    Spilled Records=2
    Shuffled Maps =1
    Failed Shuffles=0
    Merged Map outputs=1
    GC time elapsed (ms)=7
    Total committed heap usage (bytes)=468713472
Shuffle Errors
    BAD_ID=0
    CONNECTION=0
    IO_ERROR=0
    WRONG_LENGTH=0
    WRONG_MAP=0
    WRONG_REDUCE=0
File Input Format Counters 
    Bytes Read=82
File Output Format Counters 
    Bytes Written=97
Process finished with exit code 0

    public class Map extends Mapper<LongWritable, Text, Text, 
        IntWritable>{

    @Override
    public void map(LongWritable key, Text value, Context context)
            throws IOException, InterruptedException {

        String line = value.toString();
        String[] datas = line.split("\t");

            for(String data: datas) {
                Text outputKey = new Text(data);
                IntWritable outputValue = new IntWritable();
                context.write(outputKey, outputValue);
            }
    }
    }

public class Reduce extends Reducer<Text, IntWritable, Text, 
    IntWritable> {

    @Override
    public void reduce(final Text outputKey,
                       final Iterable<IntWritable> values,
                       final Context context)
            throws IOException, InterruptedException {

        int sum = 0;
        for(IntWritable value : values)
        {
            sum += value.get();
        }
        context.write(outputKey, new IntWritable(sum));
    }
}

Answer 1

您没有设置要在映射器中发出的任何IntWritable值：

IntWritable outputValue = new IntWritable();

需要替换为：

IntWritable outputValue = new IntWritable(1);

Map reduce作业执行但不产生输出

1 个答案: