Question

我有3台Centos 7服务器（禁用了防火墙和selinux） chadoop1（主服务器），chadoop2（从属服务器）和chadoop3（从属主机）

当我启动服务时，节点没有出现，我在jps上看到，没有显示DataNode和NodeManager。

所有配置在节点（从节点除外）上都是rsync

我尝试重新格式化，显示确定，但是同样的问题。

我的目录是：/ opt / hadoop

配置：

hdfs-site.xml

<configuration>
    <property>
            <name>dfs.data.dir</name>
            <value>/opt/hadoop/dfs/name/data</value>
            <final>true</final>
    </property>
    <property>
            <name>dfs.name.dir</name>
            <value>/opt/hadoop/dfs/name</value>
            <final>true</final>
    </property>
    <property>
            <name>dfs.replication</name>
            <value>2</value>
    </property>

core-site.xml

<configuration>

<property>
    <name>fs.defaultFS</name>
    <value>hdfs://localhost:8020/</value>
    <description>NameNode URI</description>
</property>

<property>
  <name>io.file.buffer.size</name>
  <value>131072</value>
  <description>Buffer size</description>
</property>

mapred-site.xml

<configuration>

<property>
    <name>mapreduce.framework.name</name>
    <value>yarn</value>
    <description>MapReduce framework name</description>
</property>

<property>
  <name>mapreduce.jobhistory.address</name>
  <value>localhost:10020</value>
  <description>Default port is 10020.</description>
</property>

<property>
  <name>mapreduce.jobhistory.webapp.address</name>
  <value>localhost:19888</value>
  <description>Default port is 19888.</description>
</property>

<property>
  <name>mapreduce.jobhistory.intermediate-done-dir</name>
  <value>/mr-history/tmp</value>
  <description>Directory where history files are written by MapReduce jobs.</description>
</property>

<property>
  <name>mapreduce.jobhistory.done-dir</name>
  <value>/mr-history/done</value>
  <description>Directory where history files are managed by the MR JobHistory Server.</description>
</property>

yarn-site.xml

<configuration>

<property>
    <name>mapreduce.framework.name</name>
    <value>yarn</value>
    <description>MapReduce framework name</description>
</property>

<property>
  <name>mapreduce.jobhistory.address</name>
  <value>localhost:10020</value>
  <description>Default port is 10020.</description>
</property>

<property>
  <name>mapreduce.jobhistory.webapp.address</name>
  <value>localhost:19888</value>
  <description>Default port is 19888.</description>
</property>

<property>
  <name>mapreduce.jobhistory.intermediate-done-dir</name>
  <value>/mr-history/tmp</value>
  <description>Directory where history files are written by MapReduce jobs.</description>
</property>

<property>
  <name>mapreduce.jobhistory.done-dir</name>
  <value>/mr-history/done</value>
  <description>Directory where history files are managed by the MR JobHistory Server.</description>
</property>

从属（仅在主服务器上，在从属服务器上有localhost）

chadoop3
chadoop4

启动服务

[hadoop@chadoop1 hadoop]$ start-dfs.sh
 Starting namenodes on [localhost]
 localhost: starting namenode, logging to /opt/hadoop/logs/hadoop-hadoop- 
 namenode-chadoop1.out
 chadoop4: starting datanode, logging to /opt/hadoop/logs/hadoop-hadoop- 
 datanode-chadoop4.out
 chadoop3: starting datanode, logging to /opt/hadoop/logs/hadoop-hadoop-                    
 datanode-chadoop3.out
 Starting secondary namenodes [0.0.0.0]
 0.0.0.0: starting secondarynamenode, logging to /opt/hadoop/logs/hadoop- 
 hadoop-secondarynamenode-chadoop1.out

 [hadoop@chadoop1 hadoop]$ jps
 5603 Jps
 5492 SecondaryNameNode
 5291 NameNode
 [hadoop@chadoop1 hadoop]$ start-yarn.sh
 starting yarn daemons
 starting resourcemanager, logging to /opt/hadoop/logs/yarn-hadoop-               
 resourcemanager-chadoop1.out
 chadoop3: starting nodemanager, logging to /opt/hadoop/logs/yarn-hadoop- 
 nodemanager-chadoop3.out
 chadoop4: starting nodemanager, logging to /opt/hadoop/logs/yarn-hadoop- 
 nodemanager-chadoop4.out
 [hadoop@chadoop1 hadoop]$ jps
 5492 SecondaryNameNode
 5658 ResourceManager
 5914 Jps
 5291 NameNode

Answer 1

所有配置在节点（从节点除外）上都是rsync

所有配置必须位于所有节点上。

也就是说，数据节点需要知道NameNode在网络上的什么位置，因此如果服务器实际上应该是从属服务器，则该进程不能在localhost上。因此，您必须输入实际的主机名。

与YARN服务相同。

我在jps上看到，没有显示DataNode和NodeManager。

从显示的输出中，您似乎只在主计算机上启动了服务，而没有两个将存在这些服务的从属服务器。

启动脚本仅控制一台计算机，而不控制集群，jps仅显示本地计算机的Java进程

顺便说一句，Apache Ambari使安装和管理Hadoop集群变得更加容易。

Hadoop 2.9多节点

1 个答案: