Question

我想运行hdfs.init()，我的代码是：

Sys.setenv(HADOOP_CMD="C:\\hdp\\hadoop\\hadoop-1.2.0.1.3.0.0-0380\\bin\\hadoop")
Sys.setenv(HADOOP_CONF_DIR="C:\\hdp\\hadoop\\hadoop-1.2.0.1.3.0.0-0380\\conf")
library(rhdfs)
hdfs.init()

执行hdfs.init()后，我收到如下错误：

> hdfs.init()
Error in .jnew("org/apache/hadoop/conf/Configuration") : 
  java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.conf.Configuration

Answer 1

下载此jar并尝试：

Sys.setenv(HADOOP_CMD="C:\\hdp\\hadoop\\hadoop-1.2.0.1.3.0.0-0380\\bin\\hadoop")
Sys.setenv(HADOOP_CONF_DIR="C:\\hdp\\hadoop\\hadoop-1.2.0.1.3.0.0-0380\\conf")
.jinit()
.jaddClassPath("/path/to/jar/hadoop-core-0.20.2-737.jar")
library(rhdfs)
hdfs.init()

确保在R

中加载java

Answer 2

您可能需要查看JAVA_HOME和HADOOP_CLASSPATH。在一个痛苦的星期天早晨之后，这个conf（尽管可能是多余的）在我的MacOS上工作:-) - 您需要根据您的机器，安装和操作系统进行必要的更改，但至少您有一个线索。

Sys.setenv ("JAVA_HOME"="/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre")
Sys.setenv ("HADOOP_PREFIX"="/usr/local/Cellar/hadoop/2.8.1")
Sys.setenv ("HADOOP_CMD"="/usr/local/Cellar/hadoop/2.8.1/bin/hadoop")
Sys.setenv ("HADOOP_STREAMING"="/usr/local/Cellar/hadoop/2.8.1/libexec/share/hadoop/tools/lib/hadoop-streaming-2.8.1.jar")
Sys.setenv ("HADOOP_CLASSPATH"="/usr/local/Cellar/hadoop/2.8.1/libexec/etc/hadoop:/usr/local/Cellar/hadoop/2.8.1/libexec/share/hadoop/common/lib/*:/usr/local/Cellar/hadoop/2.8.1/libexec/share/hadoop/common/*:/usr/local/Cellar/hadoop/2.8.1/libexec/share/hadoop/hdfs:/usr/local/Cellar/hadoop/2.8.1/libexec/share/hadoop/hdfs/lib/*:/usr/local/Cellar/hadoop/2.8.1/libexec/share/hadoop/hdfs/*:/usr/local/Cellar/hadoop/2.8.1/libexec/share/hadoop/yarn/lib/*:/usr/local/Cellar/hadoop/2.8.1/libexec/share/hadoop/yarn/*:/usr/local/Cellar/hadoop/2.8.1/libexec/share/hadoop/mapreduce/lib/*:/usr/local/Cellar/hadoop/2.8.1/libexec/share/hadoop/mapreduce/*:/contrib/capacity-scheduler/*.jar")

在rhdfs中获取hdfs.init（）中的错误

2 个答案: