即使是所需的类,Mapreduce作业也会失败

时间:2014-06-17 05:49:28

标签: hadoop mapreduce

嗨我的工作因运行时异常而失败,说没有找到mapper类。

以下是例外情况:

14/06/16 05:52:56 INFO mapred.JobClient: Task Id : attempt_201406071432_1142_m_000028_0, Status : FAILED
java.lang.RuntimeException: java.lang.ClassNotFoundException: Class com.cloudera.sa.omniture.mr.OmnitureRawDataMapper not found
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1774)
        at org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass(JobContextImpl.java:191)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:631)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)
        at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:396)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
        at org.apache.hadoop.mapred.Child.main(Child.java:262)
Caused by: java.lang.ClassNotFoundException: Class com.cloudera.sa.omniture.mr.OmnitureRawDataMapper not found
        at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1680)
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1772)
        ... 8 more

但是在我的jar中我曾经运行过Mapper类的工作,我仍然面临着这个问题。请指导。

以下信息。在我的jar文件中确认jar文件中存在Mapper类。

[svtdphpd@d-hcr75y1 testJARs]$ jar tvf MalformedAnalysis.jar | grep Omniture
  1405 Mon Jun 16 15:59:34 CDT 2014 com/cloudera/sa/omniture/mr/OmnitureRawDataMapper$HITDATA_PROBLEM.class
  3728 Mon Jun 16 15:59:34 CDT 2014 com/cloudera/sa/omniture/mr/OmnitureRawDataMapper.class
  5280 Mon Jun 16 16:00:46 CDT 2014 com/cloudera/sa/omniture/mr/OmnitureToRCFileJob.class
  6436 Mon Jun 16 16:03:34 CDT 2014 com/cloudera/sa/omniture/mr/OmnitureDataFileRecordReader.class
  3642 Mon Jun 16 16:00:16 CDT 2014 com/cloudera/sa/omniture/mr/OmnitureRawDataReducer.class
  1792 Mon Jun 16 15:59:34 CDT 2014 com/cloudera/sa/omniture/mr/OmnitureDataFileInputFormat.class

以下是作业配置:

// Create job
Job job = Job.getInstance(config, "LoadOmnitureData");
job.setJarByClass(OmnitureToRCFileJob.class);

// Add named output for malformed records

MultipleOutputs.addNamedOutput(job, "Malformed",
        TextOutputFormat.class, NullWritable.class, Text.class);

FileInputFormat.addInputPath(job, new Path(inputPath));

job.setInputFormatClass(OmnitureDataFileInputFormat.class);     
job.setOutputFormatClass(TextOutputFormat.class);

job.setOutputKeyClass(NullWritable.class);
job.setOutputValueClass(Text.class);

job.setReducerClass(OmnitureRawDataReducer.class);
job.setMapperClass(OmnitureRawDataMapper.class);

job.setMapOutputKeyClass(Text.class);
job.setMapOutputValueClass(Text.class);

TextOutputFormat.setOutputPath(job, new Path(outputPath));

job.setNumReduceTasks(numReduceTasks);

0 个答案:

没有答案