Question

我在几台笔记本电脑中安装了hadoop以形成一个hadoop集群。首先我们以伪分布式模式安装，除了一个非常完美（即所有服务都运行，当我使用hadoop fs进行测试时，它显示hdfs）。在后面提到的笔记本电脑（有问题的笔记本电脑）中，`hadoop fs -ls命令显示本地目录的信息而不是 hdfs，命令-cat也是如此。，-mkdir，-put。我能做错什么？

任何帮助将不胜感激

这是我的core-site.xml

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->

<configuration>
<property>
 <name>hadoop.tmp.dir</name>
 <value>/home/hduser/hdfs_dir/tmp</value>
 <description></description>
</property>

<property>
 <name>fs.default.name</name>
 <value>hdfs://localhost:54310</value>
 <description>.</description>
</property>
</configuration>

我必须说，这是所有其他笔记本电脑的相同文件，它们工作正常。

Answer 1

我遇到了同样的问题，我必须确保fs.default.name的值包含一个尾随/来引用路径组件：

<property>
 <name>fs.default.name</name>
 <value>hdfs://localhost:54310/</value>
 <description>.</description>
</property>

Answer 2

检查fs.default.name中的core-site.xml是否指向正确的datanode 在前：

<property>
     <name>fs.default.name</name>
     <value>hdfs://target-namenode:54310</value>
</property>

Answer 3

如果fs.default.name中的core-site.xml指向hdfs://localhost:54310/，有或没有尾随/，即使您遇到同样的问题，那么您可能会查看错误的配置文件。在我的情况下，它是cloudera的cdh4并检查符号链接：

ls -l /etc/hadoop/conf ** /etc/hadoop/conf -> /etc/alternatives/hadoop-conf ls -l /etc/alternatives/hadoop-conf ** /etc/alternatives/hadoop-conf -> /etc/hadoop/conf.cloudera.yarn1

之前我使用MRv1并迁移到MRv2（YARN），升级后sym链接被破坏为：

ls -l /etc/hadoop/conf ** /etc/hadoop/conf -> /etc/alternatives/hadoop-conf ls -l /etc/alternatives/hadoop-conf ** /etc/alternatives/hadoop-conf -> /etc/hadoop/conf.cloudera.mapreduce1 ls -l /etc/hadoop/conf.cloudera.mapreduce1 ls: cannot access /etc/hadoop/conf.cloudera.mapreduce1: No such file or directory



Also, update-alternatives was run to have high priority for /etc/hadoop/conf.cloudera.mapreduce1 path as:



alternatives --display hadoop-conf

hadoop-conf - status is manual.

 link currently points to /etc/hadoop/conf.cloudera.mapreduce1

/etc/hadoop/conf.cloudera.hdfs1 - priority 90

/etc/hadoop/conf.cloudera.mapreduce1 - priority 92

/etc/hadoop/conf.empty - priority 10

/etc/hadoop/conf.cloudera.yarn1 - priority 91

Current `best' version is /etc/hadoop/conf.cloudera.mapreduce1.



To remove old link which has highest priority do:

alternatives --display hadoop-conf hadoop-conf - status is manual. link currently points to /etc/hadoop/conf.cloudera.mapreduce1 /etc/hadoop/conf.cloudera.hdfs1 - priority 90 /etc/hadoop/conf.cloudera.mapreduce1 - priority 92 /etc/hadoop/conf.empty - priority 10 /etc/hadoop/conf.cloudera.yarn1 - priority 91 Current `best' version is /etc/hadoop/conf.cloudera.mapreduce1.

hadoop fs命令显示本地文件系统而不是hdfs

3 个答案: