获取映射器函数hadoop

时间:2015-12-09 07:19:40

标签: java hadoop mapreduce bigdata hadoop2

我的输入如下:

0RViGi2Rne8,OmnipotentSI,1135,Entertainment,9,490,2.31,26,15
HT_QlOJbDpg,dumbasstheredneck,1095,Comedy,14,1349,2.09,22,13
ShhClb6J-NA,thecrashguy,1136,Entertainment,61,29,0,0,0
g9e1alirMhc,iamdanielisaac,541,Comedy,152,483053,4.65,2689,2932
YZev1imoxX8,threefirstnames,1135,Entertainment,253,63,5,3,2

我想要我的Mapper输出。我基本上希望第4列成为我的Key,我只希望整数1作为每个键的值:

Entertainment: 1
Comedy: 1
Entertainment: 1
Comedy: 1
Entertainment: 1

我的mapper类如下所示。为此,我得到java.lang.ArrayIndexOutOfBoundsException:3。我无法理解为什么,因为在正常的java程序上测试时,我的split函数返回了正确的输出。

public class VidCountByCategory {
    public static class Map extends Mapper<LongWritable,Text,Text,IntWritable>{
        public void map(LongWritable key, Text value,Context context)
                throws IOException,InterruptedException {

            String line = value.toString();
            String category = line.split(",")[3];

                value.set(category);
                context.write(value, new IntWritable(1));
        }

    }

任何人都可以指出我哪里出错了吗?

0 个答案:

没有答案