Question

        I created a hadoop multinode cluster and my target is to know processor information of all nodes in the cluster by running jar file in master.I have the code to know processor info in a node.How can i run this code in all nodes and retrieve the processor information using java.
        thanks in advance for help..

        Driver code:

    Here i have used only mapper class    

       public class ProcInfoCluster extends Configured implements Tool {

            @Override
            public int run(String[] arg0) throws Exception {
                // TODO Auto-generated method stub
                if(arg0.length<2)
                {
                    System.out.println("please provide the output file paths");
                    return -1;
                }

                JobConf conf=new JobConf(ProcInfoCluster.class);
                conf.setJobName("ProcesorInfo");
                FileInputFormat.setInputPaths(conf, new Path(arg0[0]));
                FileOutputFormat.setOutputPath(conf,new Path(arg0[1]));
                conf.setMapperClass(ProcInfoMapper.class);
                //conf.setReducerClass(WordReducer.class);
                conf.setMapOutputKeyClass(Text.class);
                conf.setMapOutputValueClass(Text.class);
                conf.setOutputKeyClass(Text.class);
                conf.setOutputValueClass(Text.class);
                JobClient.runJob(conf);     
                return 0;
            }
            public static void main(String args[])
            throws Exception
            {
            int exitCode=ToolRunner.run(new ProcInfoCluster(), args);   
            System.exit(exitCode);
            }


        }



        Mapper code:


    Here in this mapper code i have written code to know processor information and tried to send it as key value pairs

        public class ProcInfoMapper extends MapReduceBase implements Mapper<LongWritable, Text, Text,Text>
        {

            @Override

// map函数检索所有节点的处理器信息 public void map（LongWritable key，Text value， OutputCollector输出，记者记者）抛出IOException { // TODO自动生成的方法存根

                String no=System.getenv("PROCESSOR_ARCHITEW6432");
    //to know processor info
                System.out.println(no);
                String s=new String("ProcInfo");
                output.collect(new Text(s), new Text(no));
            }

        }

and i had a doubt that whether we can write a map reduce application only with mapper class without the reducer class

如何在hadoop集群节点中执行一段代码？

0 个答案: