给定以这种方式构建的数据集,我正在尝试解决任务要获取的mapreduce问题: 节点x节点y
2 4
3 1
. .
. .
只有与x存在(x,y)的y
输出应该类似于:
2
3
问题是,当我尝试执行应用程序时,出现此错误:
biar@pfp-VirtualBox:~$ yarn jar /home/biar/Desktop/examgraph.jar ExamGraph 3 12
Exception in thread "main" java.lang.NoSuchMethodException: ExamGraph.main([Ljava.lang.String;)
at java.lang.Class.getMethod(Class.java:1678)
at org.apache.hadoop.util.RunJar.run(RunJar.java:215)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
这是我的应用程序的代码:
public class ExamGraph {
public static class TokenizeMapper extends Mapper<LongWritable , Text, Text, Text>{
public void map(LongWritable key, Text value, Context context
) throws IOException, InterruptedException {
int[] a = new int[2];
value = new Text(value.toString());
String[] coppia = value.toString().split(" ");
a[0] = Integer.parseInt(coppia[0]);
a[1] = Integer.parseInt(coppia[1]);
if (a[1] > a[0]) context.write(new Text(coppia[1]), new Text("SI,0"));
else context.write(new Text(coppia[1]), new Text("NO,0"));
String[] coppiaInv = value.toString().split(" ");
a[0] = Integer.parseInt(coppiaInv[1]);
a[1] = Integer.parseInt(coppiaInv[0]);
if (a[0] < a[1]) context.write(new Text(coppiaInv[0]), new Text("SI,1"));
else context.write(new Text(coppiaInv[0]), new Text("NO,1"));
}
}
public static class IntSumReducer
extends Reducer<Text,Text,Text,NullWritable> {
public void reduce(Text key, Iterable<Text> values,
Context context
) throws IOException, InterruptedException {
boolean evenResult = false;
boolean oddResult = false;
for (Text value : values) {
String[] valori = value.toString().split(",");
int a;
a = Integer.parseInt(valori[1]);
if (valori[0].equals("SI") && a == 0) evenResult = true;
if (valori[0].equals("SI") && a == 1) oddResult = true;
}
if (evenResult == true && oddResult == true) {
context.write(key, NullWritable.get());
}
else context.write(new Text("Not valid"), NullWritable.get());
}
}
public static void main(String[] args) throws Exception {
Configuration conf = new Configuration();
Job job = Job.getInstance(conf, "ExamGraph");
job.setJarByClass(ExamGraph.class);
job.setMapperClass(TokenizeMapper.class);
job.setCombinerClass(IntSumReducer.class);
job.setReducerClass(IntSumReducer.class);
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(NullWritable.class);
FileInputFormat.addInputPath(job, new Path(args[0]));
FileOutputFormat.setOutputPath(job, new Path(args[1]));
System.exit(job.waitForCompletion(true) ? 0 : 1);
}
}
我认为问题出在主代码中,但我看不出它在哪里。 有人提示吗?