Mapper代码处理空间也是如此

时间:2017-10-22 15:35:23

标签: java hadoop mapreduce bigdata

我正在尝试从映射器中的输入文件中读取记录,其中我不想处理那些具有空格的记录。不幸的是,即使是有空格的记录也会得到处理。你能告诉我吗?

代码:

  @Override
    protected void setup(Context context) throws IOException, InterruptedException {
        BufferedReader bufferedReader = new BufferedReader(new FileReader("smallData.csv"));
        header = bufferedReader.readLine();
        headerList = header.split(",");
    }

public void map(LongWritable key, Text value, Context context) throws IOException,InterruptedException {

            String line = value.toString();
            String[] args = line.split(",");

            if(headerList.length == args.length && !header.equals(line)) 
            {
            String srcip = args[4];
            String dstip = args[5];
            String scountry = args[20];
            String dcountry = args[22];
            Text sourceIP = new Text();
            Text sourceCountry = new Text();
            Text destIP = new Text();
            Text destCountry= new Text();

            if(!scountry.equals(""))
            {   
                sourceIP.set(srcip);
                sourceCountry.set(scountry);
                context.write(sourceCountry,sourceIP);

           }

            if(!dcountry.equals(""))
             {  
                destIP.set(dstip); 
                destCountry.set(dcountry);
                context.write(destCountry,destIP);
             }

0 个答案:

没有答案