在Hadoop上启动的机器学习h2o无法正常工作

时间:2019-09-09 10:22:10

标签: hadoop h2o

我需要从Hadoop启动机器学习套件H2O,但是我收到以下错误消息,根据h2o starting on YARN not working修改yarn参数,但仍然失败

错误日志:

  

错误:无法启动任何H2O节点;请与您的YARN管理员联系。

一个常见的原因是请求的容器大小(1.1 GB)超过了以下YARN设置:

yarn.nodemanager.resource.memory-mb
yarn.scheduler.maximum-allocation-mb

yarn-site.xml:

<configuration>

<!-- Site specific YARN configuration properties -->
           <property>
          <name>yarn.nodemanager.aux-services</name>
              <value>mapreduce_shuffle</value>
       </property>
       <property>
          <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
          <value>org.apache.hadoop.mapred.ShuffleHandler</value>
       </property>
       <property>
              <name>yarn.resourcemanager.scheduler.address</name>
              <value>master:8030</value>
           </property>
       <property>
          <name>yarn.resourcemanager.resource-tracker.address</name>
          <value>master:8025</value>
       </property>
       <property>
          <name>yarn.resourcemanager.address</name>
          <value>master:8050</value>
           </property>        
       <property>
           <name>yarn.scheduler.maximum-allocation-mb</name>
           <value>16135</value>

       </property>
       <property>
           <name>yarn.nodemanager.resource.memory-mb</name>
           <value>16135</value>
       </property>
       <property>
           <name>yarn.scheduler.maximum-allocation-vcores</name>
           <value>8</value>
       </property>
       <property>
           <name>yarn.nodemanager.resource.cpu-vcores</name>
           <value>4</value>

       </property>  
</configuration>

0 个答案:

没有答案