Question

我正在尝试在hadoop上运行h2o：

hadoop 2.7.1（单节点）
h2o driver - h2o-3.10.4.3-mapr5.1

使用以下命令：

hadoop jar h2odriver.jar -nodes 1 -mapperXmx 6g -output hdfsOutputDirName

并收到此错误：

----- YARN cluster metrics -----
Number of YARN worker nodes: 1

----- Nodes -----
Node: http://DESKTOP-BPF0VCF:8042 Rack: /default-rack, RUNNING, 0 containers used, 0,0 / 16,0 GB used, 0 / 4 vcores used

----- Queues -----
Queue name:            default
    Queue state:       RUNNING
    Current capacity:  0,00
    Capacity:          1,00
    Maximum capacity:  1,00
    Application count: 0

Queue 'default' approximate utilization: 0,0 / 16,0 GB used, 0 / 4 vcores used

----------------------------------------------------------------------

ERROR: Unable to start any H2O nodes; please contact your YARN administrator.

       A common cause for this is the requested container size (6,6 GB)
       exceeds the following YARN settings:

           yarn.nodemanager.resource.memory-mb
           yarn.scheduler.maximum-allocation-mb

----------------------------------------------------------------------

For YARN users, logs command is 'yarn logs -applicationId application_1492203365290_0001'

这是我的yarn-site.xml：

<configuration>
   <property>
       <name>yarn.nodemanager.aux-services</name>
       <value>mapreduce_shuffle</value>
   </property>
   <property>
       <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
       <value>org.apache.hadoop.mapred.ShuffleHandler</value>
   </property>
      <property>
       <name>yarn.resourcemanager.scheduler.class</name>
       <value>org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler</value>
   </property>

   <property>
       <name>yarn.scheduler.maximum-allocation-mb</name>
       <value>16384</value>
   </property>

   <property>
       <name>yarn.nodemanager.resource.memory-mb</name>
       <value>16384</value>
   </property>

   <property>
       <name>yarn.scheduler.maximum-allocation-vcores</name>
       <value>8</value>
   </property>

   <property>
       <name>yarn.nodemanager.resource.cpu-vcores</name>
       <value>4</value>
   </property>
   
</configuration>

所以，很明显内存已经足够了，但它仍然不起作用。这篇文章也没有帮助： h2o starting on YARN not working

hadoop集群上的h2o不起作用

0 个答案: