我正在尝试在MongoDB连接器中为hadoop提供的Treasury Yield示例。我正在关注此链接:
http://docs.mongodb.org/ecosystem/tutorial/getting-started-with-hadoop/
实际上,我在/ usr / local / hadoop中安装了hadoop2.4.0。然后,我按照/ usr / local / hadoop / lib /中的解释编译并复制了所有jar(驱动程序,核心,流媒体......)。
启动我的hadoop集群后,以及运行时 ./gradlew jar testJar historicalYield
这是我得到的错误:
Updating mongo jars
Download http://central.maven.org/maven2/org/mongodb/mongo-java-driver/2.12.2/mongo-java-driver-2.12.2.jar
Updating cluster configuration
:configureCluster
Configuring 2.4 cluster
Shutting down node manager
Shutting down resource manager
Shutting down data node
Shutting down name node
build/hadoop-2.4.sh: line 36: $LOG: ambiguous redirect
Starting namenode
Starting datanode
Starting resourcemanager
Starting nodemanager
Starting hiveserver
:historicalYield
connected to: 127.0.0.1
2014-12-27T13:18:15.660+0100 dropping: mongo_hadoop.yield_historical.in
2014-12-27T13:18:15.924+0100 check 9 5193
2014-12-27T13:18:15.938+0100 imported 5193 objects
Executing hadoop job:
/home/hduser/hadoop-binaries/hadoop-2.4.0/bin/hadoop \
jar \
examples/treasury_yield/build/libs/treasury_yield-1.3.0.jar \
com.mongodb.hadoop.examples.treasury.TreasuryYieldXMLConfig \
-Dmongo.input.split_size=8 \
-Dmongo.job.verbose=true \
-Dmongo.input.uri=mongodb://localhost:27017/mongo_hadoop.yield_historical.in \
-Dmongo.output.uri=mongodb://localhost:27017/mongo_hadoop.yield_historical.out
Exception in thread "main" java.lang.NoClassDefFoundError: com/mongodb/hadoop/util/MongoTool
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:274)
at org.apache.hadoop.util.RunJar.main(RunJar.java:205)
caused by: java.lang.ClassNotFoundException: com.mongodb.hadoop.util.MongoTool
一句话:在执行示例时,我觉得在本例的复杂化过程中,我手动编写并复制到/ usr / local / hadoop / lib下的drvier没有链接。实际上,在执行./gradlew命令时,gradlew再次下载hadoop和另一个版本的驱动程序,如下所示:
:streaming:testClasses UP-TO-DATE
:streaming:testJar
:installHadoop
Extracting /home/hduser/hadoop-binaries/hadoop-2.4.0.tar.gz
:installHive
Extracting /home/hduser/hadoop-binaries/hive-0.12.0.tar.gz
:copyFiles
Updating mongo jars
Download http://central.maven.org/maven2/org/mongodb/mongo-java-driver/2.12.2/mongo-java-driver-2.12.2.jar
任何帮助都将不胜感激。