使用gora0.5和mongodb作为后端数据存储区,无法在hadoop2.4.0上运行nutch2.3-snapshot

时间:2014-11-23 03:38:52

标签: hadoop nutch gora

我遇到这个问题几天了。 当我使用hadoop1.2时,它可以正常工作。 当我转向hadoop2.x(hadoop2.4或hadoop2.5.2)时,我遇到了这个问题:

java.lang.Exception: java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.TaskAttemptContext, but class was expected
    at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462)
    at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:522)
Caused by: java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.TaskAttemptContext, but class was expected
    at org.apache.gora.mapreduce.GoraOutputFormat.getRecordWriter(GoraOutputFormat.java:83)
    at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.<init>(MapTask.java:624)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:744)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
    at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:744)

我发现当我在ivy.xml中使用hadoop2.x时,它会自动生成hadoop-core-1.0.1.jar,它似乎受到gora依赖性的影响。排除hadoop-core- *之后,会出现这个问题! 我还手动将jar文件avro-mapre-1.7.6.jar更新为avro-mapred-1.7.6-hadoop2.jar,但遗憾的是没有任何变化! 感谢任何想法,谢谢!

0 个答案:

没有答案