Question

编辑：我查看了YARN Resourcemanager not connecting to nodemanager，解决方案对我不起作用。我已经附加了节点管理器日志的一部分，其中建立了与资源管理器的连接：

[main] client.RMProxy (RMProxy.java:createRMProxy(98)) - Connecting to ResourceManager at /0.0.0.0:8031

2016-06-17 19:01:04,697 INFO  [main] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNMContainerStatuses(429)) - Sending out 0 NM container statuses: []

2016-06-17 19:01:04,701 INFO  [main] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:registerWithRM(268)) - Registering with RM using containers :[]

2016-06-17 19:01:05,815 INFO  [main] ipc.Client (Client.java:handleConnectionFailure(867)) - Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 0 time(s); retry policy is     RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)

2016-06-17 19:01:06,816 INFO  [main] ipc.Client (Client.java:handleConnectionFailure(867)) - Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)

由于某种原因，它说它连接到0.0.0.0。当我ssh到其中一个数据节点并ping资源管理器时，我得到一个响应，因此它能够解析主机名。

这让我相信我的yarn-site.xml中的选项不正确，因为我的节点正在尝试连接到0.0.0.0:8031而不是资源管理器：8031

我在dockers上运行Cloudera hadoop集群，并且遇到Yarn资源管理器能够看到其他节点的问题。它们的设置方式如下：

Node1 - Namenode（hadoop-hdfs-namenode）

节点2 - 辅助Namenode（hadoop-hdfs-secondarynamenode）

节点3 - 纱线资源管理器（hadoop-yarn-resourcemanager）

节点4 - datanode和节点管理器（hadoop-hdfs-datanode，hadoop-yarn-nodemanager）

节点5 - datanode和节点管理器（hadoop-hdfs-datanode，hadoop-yarn-nodemanager）

当我去namenode：50070时，我能够看到两个节点。但是，当我去资源管理器时：8088它显示我有零节点。我的yarn-site.xml文件位于每个节点上，如下所示：

<configuration>
  <property>
    <name>yarn.resourcemanager.address</name>
    <value>resource-manager:8032</value>
  </property>
  <property>
    <name>yarn.resourcemanager.scheduler.address</name>
    <value>resource-manager:8030</value>
  </property>
  <property>
    <description>Classpath for typical applications.</description>
    <name>yarn.application.classpath</name>
    <value>
      $HADOOP_CONF_DIR,
      $HADOOP_COMMON_HOME/*,$HADOOP_COMMON_HOME/lib/*,
      $HADOOP_HDFS_HOME/*,$HADOOP_HDFS_HOME/lib/*,
      $HADOOP_MAPRED_HOME/*,$HADOOP_MAPRED_HOME/lib/*,
      $HADOOP_YARN_HOME/*,$HADOOP_YARN_HOME/lib/*
    </value>
  </property>
  <property>
    <name>yarn.nodemanager.aux-services</name>
    <value>mapreduce_shuffle</value>
  </property>
  <property>
    <name>yarn.nodemanager.local-dirs</name>
    <value>file:///data/1/yarn/local,file:///data/2/yarn/local,file:///data/3/yarn/local</value>
  </property>
  <property>
    <name>yarn.nodemanager.log-dirs</name>
    <value>file:///data/1/yarn/logs,file:///data/2/yarn/logs,file:///data/3/yarn/logs</value>
  </property>
  <property>
    <name>yarn.log.aggregation-enable</name>
    <value>true</value>
  </property>
  <property>
    <description>Where to aggregate logs</description>
    <name>yarn.nodemanager.remote-app-log-dir</name>
    <value>hdfs://namenode:8020/var/log/hadoop-yarn/apps</value>
  </property>
  <property>
    <name>yarn.resourcemanager.webapp.address</name>
    <value>resource-manager:8088</value>
  </property>
  <property>
    <name>yarn.resourcemanager.resource-tracker.address</name>
    <value>resource-manager:8031</value>
  </property>
  <property>
    <name>yarn.resourcemanager.admin.address</name>
    <value>resource-manager:8033</value>
  </property>
  <property>
    <description>
    Number of seconds after an application finishes before the nodemanager's
    DeletionService will delete the application's localized file directory
    and log directory.

    To diagnose Yarn application problems, set this property's value large
    enough (for example, to 600 = 10 minutes) to permit examination of these
    directories. After changing the property's value, you must restart the
    nodemanager in order for it to have an effect.

    The roots of Yarn applications' work directories is configurable with
    the yarn.nodemanager.local-dirs property (see below), and the roots
    of the Yarn applications' log directories is configurable with the
    yarn.nodemanager.log-dirs property (see also below).
    </description>
    <name>yarn.nodemanager.delete.debug-delay-sec</name>
    <value>600</value>
  </property>
</configuration>

有没有人知道为什么会这样？

感谢阅读。

Answer 1

指定：

<property>
  <name>yarn.resourcemanager.hostname</name>
  <value>master-1</value>
</property>

Answer 2

如编辑中所示，似乎没有拾取yarn-site.xml，只发生了默认值。我解决了这个问题，将yarn-site.xml文件以root用户身份复制到计算机上的每个目录中。然后我运行节点管理器，因为它不能在用户root下运行，因此读取文件时出错。该日志将我引导到预期文件位于纱线特定目录而不是通用hadoop目录的位置。

资源管理器没有节点

2 个答案: