Question

我试图在纱线上使用cgroup。

但我遇到了一些麻烦。

我使用1个namenod和3个datanode。

我无法运行容器执行程序。

我认为我做错了设置属性，但我不知道

你帮我吗？如果您需要更多信息，我会写评论。

纱-site.xml中

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. See accompanying LICENSE file.
-->
<configuration>
<property>
<name>yarn.resourcemanager.scheduler.class</name>
<value>org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler</value>
</property>
<property>
<name>yarn.scheduler.fair.allocation.file</name>
<value>/home/hadoop/tools/hadoop/etc/hadoop/fair-scheduler.xml</value>
</property>
<property>
<name>yarn.resourcemanager.hostname</name>
<value>em-name01</value>
</property>
<property>
<name>yarn.nodemanager.resource.memory-mb</name>
<value>8192</value>
</property>
<property>
<name>yarn.nodemanager.resource.cpu-vcores</name>
<value>4</value>
</property>
<property>
<name>yarn.nodemanager.resource.percentage-physical-cpu-limit</name>
<value>100</value>
</property>

<property>
<name>yarn.scheduler.fair.allow-undeclared-pools</name>
<value>false</value>
</property>
<property>
<name>yarn.scheduler.fair.preemption</name>
<value>false</value>
</property>


<!-- cgroup config -->
<property>
<name>yarn.nodemanager.container-executor.class</name>
<value>org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor</value>
</property>
<property>
<name>yarn.nodemanager.linux-container-executor.nonsecure-mode.limit-users</name>
<value>false</value>
</property>
<property>
<name>yarn.nodemanager.linux-container-executor.resources-handler.class</name>
<value>org.apache.hadoop.yarn.server.nodemanager.util.CgroupsLCEResourcesHandler</value>
</property>
<property>
<name>yarn.nodemanager.linux-container-executor.cgroups.hierarchy</name>
<value>/hadoop-yarn</value>
</property>
<property>
<name>yarn.nodemanager.linux-container-executor.cgroups.mount</name>
<value>true</value>
</property>
<property>
<name>yarn.nodemanager.linux-container-executor.cgroups.mount-path</name>
<value>/cgroup/cpu</value>
</property>
<property>
<name>yarn.nodemanager.linux-container-executor.group</name>
<value>hadoop</value>
</property>
<!--
-->
</configuration>

容器executor.cfg

yarn.nodemanager.linux-container-executor.group = hadoop＃配置的yarn.nodemanager.linux-container-executor.group值 yarn.nodemanager.local-显示目录= / TMP / Hadoop的的hadoop /纳米本地-DIR banned.users =＃逗号分隔的无法运行应用程序的用户列表 min.user.id = 500＃防止其他超级用户 allowed.system.users = hadoop ##逗号分隔的CAN运行应用程序的系统用户列表

容器执行

[hadoop @ em-name01 hadoop] $ ll $ HADOOP_HOME / bin /

--- Sr-s --- 1 root hadoop 160351 1月26日00:20容器执行者

[hadoop @ em-name01 hadoop] $ ll container-executor.cfg

-rw-r - r-- 1 root hadoop 389 May 27 02:38 container-executor.cfg

纱线Hadoop的节点管理器-EM-data02.log （所有数据节点都有相同的日志）

*************** ************* / 2016-05-27 10：45：31,586 INFO org.apache.hadoop.yarn.server.nodemanager.NodeManager：[TERM，HUP，INT]的已注册UNIX信号处理程序 2016-05-27 10：45：32,826 WARN org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor：yarn.nodemanager.lin ux-container-executor.nonsecure-mode.limit-users：未启用身份验证的模拟 2016-05-27 10：45：32,830 WARN org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor：来自container执行器初始化的退出代码是：24 ExitCodeException exitCode = 24：文件/home/hadoop/tools/hadoop-2.7.2/etc/hadoop必须由root拥有，但归500所有在org.apache.hadoop.util.Shell.runCommand（Shell.java:545）在org.apache.hadoop.util.Shell.run（Shell.java:456）在org.apache.hadoop.util.Shell $ ShellCommandExecutor.execute（Shell.java:722）在org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.init（LinuxContainerExecutor.java:185）在org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit（NodeManager.java:216）在org.apache.hadoop.service.AbstractService.init（AbstractService.java:163）在org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager（NodeManager.java:485）在org.apache.hadoop.yarn.server.nodemanager.NodeManager.main（NodeManager.java:533） 2016-05-27 10：45：32,832 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor： 2016-05-27 10：45：32,832 INFO org.apache.hadoop.service.AbstractService：服务NodeManager在状态INITED失败;原因：org.apache.hadoop.yarn.exceptions.YarnRuntimeException：初始化容器执行程序失败 org.apache.hadoop.yarn.exceptions.YarnRuntimeException：初始化容器执行程序失败在org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit（NodeManager.java:218）在org.apache.hadoop.service.AbstractService.init（AbstractService.java:163）在org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager（NodeManager.java:485）在org.apache.hadoop.yarn.server.nodemanager.NodeManager.main（NodeManager.java:533）引起：java.io.IOException：Linux容器执行程序未正确配置（错误= 24）在org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.init（LinuxContainerExecutor.java:191）在org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit（NodeManager.java:216） ......还有3个引发者：ExitCodeException exitCode = 24：文件/home/hadoop/tools/hadoop-2.7.2/etc/hadoop必须由root拥有，但归500所有在org.apache.hadoop.util.Shell.runCommand（Shell.java:545）在org.apache.hadoop.util.Shell.run（Shell.java:456）在org.apache.hadoop.util.Shell $ ShellCommandExecutor.execute（Shell.java:722）在org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.init（LinuxContainerExecutor.java:185） ......还有4个 2016-05-27 10：45：32,834致命org.apache.hadoop.yarn.server.nodemanager.NodeManager：启动NodeManager时出错 org.apache.hadoop.yarn.exceptions.YarnRuntimeException：初始化容器执行程序失败在org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit（NodeManager.java:218）在org.apache.hadoop.service.AbstractService.init（AbstractService.java:163）在org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager（NodeManager.java:485）在org.apache.hadoop.yarn.server.nodemanager.NodeManager.main（NodeManager.java:533）引起：java.io.IOException：Linux容器执行程序未正确配置（错误= 24）在org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.init（LinuxContainerExecutor.java:191）在org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit（NodeManager.java:216） ......还有3个引发者：ExitCodeException exitCode = 24：文件/home/hadoop/tools/hadoop-2.7.2/etc/hadoop必须由root拥有，但由500拥有在org.apache.hadoop.util.Shell.runCommand（Shell.java:545）在org.apache.hadoop.util.Shell.run（Shell.java:456）在org.apache.hadoop.util.Shell $ ShellCommandExecutor.execute（Shell.java:722）在org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.init（LinuxContainerExecutor.java:185） ......还有4个 2016-05-27 10：45：32,838 INFO org.apache.hadoop.yarn.server.nodemanager.NodeManager：SHUTDOWN_MSG： / ******************* *********** SHUTDOWN_MSG：在em-data02 / 58.181.37.179处关闭NodeManager ************************************************** ********** /

Answer 1

试试这个：

chown "/home/hadoop/tools/hadoop-2.7.2/etc/hadoop"

root用户所有者。

yarn container-executor退出代码24 ERROR

1 个答案: