Question

我正在测试一些基本的HDFS操作，比如创建目录。我的测试中有以下群集配置：

import org.apache.hadoop.fs._
import org.apache.hadoop.fs.permission.FsPermission
import org.apache.hadoop.hdfs.{HdfsConfiguration, MiniDFSCluster}

// ...

private val baseDir = new File("./target/hdfs/test").getAbsoluteFile

private val conf = new HdfsConfiguration()
conf.set(MiniDFSCluster.HDFS_MINIDFS_BASEDIR, baseDir.getAbsolutePath)
conf.setInt("dfs.safemode.threshold.pct", 0)
private val builder = new MiniDFSCluster.Builder(conf)
private val cluster = builder.build()
cluster.waitActive()
private val fs = cluster.getFileSystem

private val host = cluster.getNameNode.getHttpAddress.getHostString
private val port = cluster.getNameNodePort

我发现当我运行测试时，我发现我总是遇到这个错误：

[warn] o.a.h.s.UserGroupInformation - PriviledgedActionException as:erip (auth:SIMPLE) cause:org.apache.hadoop.hdfs.server.namenode.SafeModeException: Cannot create directory [...]. Name node is in safe mode.
Resources are low on NN. Please add or free up more resources then turn off safe mode manually. NOTE:  If you turn off safe mode before adding resources, the NN will immediately return to safe mode. Use "hdfs dfsadmin -safemode leave" to turn safe mode off.

紧随其后......

[info]   org.apache.hadoop.ipc.RemoteException: Cannot create directory [...]. Name node is in safe mode.
[info] Resources are low on NN. Please add or free up more resources then turn off safe mode manually. NOTE:  If you turn off safe mode before adding resources, the NN will immediately return to safe mode. Use "hdfs dfsadmin -safemode leave" to turn safe mode off.

我正在运行内存中的群集，所以我不知道为什么我会看到这个。我认为设置"dfs.safemode.threshold.pct"会阻止我根据this answer看到此错误，但我错了。

为什么我的内存测试群集在安全模式下运行？我如何阻止它这样做？

Answer 1

使用以下命令退出运行任何命令的安全模式 -

String value;
ClassA a = new ClassA();
value = a.tAreaA.getText();
tAreaB.setText(value);

与safemode相关的其他命令是

hadoop dfsadmin -safemode leave

获取系统的当前运行状态

Answer 2

问题在于cluster.waitActive()，它等待名称节点准备就绪。这应该是cluster.waitClusterUp()，它明确地等待群集退出安全模式。

为什么我的测试集群在安全模式下运行？

2 个答案: