Question

我是hadoop生态系统的新手，正在尝试设置hadoop-3.1.0多节点集群。群集启动正常，所有节点和从属设备设置如下：

node-1
$ jps
3248 NodeManager
2658 DataNode
3109 ResourceManager
4748 Jps
4540 SecondaryNameNode
4301 NameNode

node-2 & node-3
$ jps
1792 DataNode
1856 Jps

即使datanode配置为在从属服务器/工作器上运行，问题仍然存在。 start-dfs.sh也正在节点1上启动datanode。请让我们知道如何解决此问题。

以下是配置文件

core-site.xml

<configuration>
  <property>
      <name>fs.default.name</name>
      <value>hdfs://hdp-1:9000</value>
  </property>
</configuration>

hdfs-site.xml

<configuration>
<property>
<name>dfs.namenode.name.dir</name>
<value>file:/home/hadoop/hadoop/hadoop_store/hdfs/namenode2</value>
</property>
<property>
<name>dfs.datanode.data.dir</name>
<value>file:/home/hadoop/hadoop/hadoop_store/hdfs/datanode2
</value>
</property>
</configuration>

mapred-site.xml

<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration>

yarn-site.xml

<configuration>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
<value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>
<property>
<name>yarn.resourcemanager.resource-tracker.address</name>
<value>hdp-1:8025</value>
</property>
<property>
<name>yarn.resourcemanager.scheduler.address</name>
<value>hdp-1:8030</value>
</property>
<property>
<name>yarn.resourcemanager.address</name>
<value>hdp-1:8050</value>
</property>
</configuration>

workers

$ cat workers
hdp-2
hdp-3

hadoop在namenode上启动datanode

0 个答案: