Question

我想删除Hadoop集群中的所有缓存文件夹。我已经写下了一些步骤，但我不确定如何继续使用Ansible脚本。总的来说，我理解Ansible，但是我对我应该使用哪些模块感到困惑。

步骤：

停止YARN服务。

$ sudo service hadoop-mapreduce-historyserver stop
$ sudo service hadoop-yarn-resourcemanager stop
$ sudo service hadoop-yarn-nodemanager stop

登录所有节点并删除用户缓存目录的内容。例如：

for i in cat list_of_nodes_in_cluster; do ssh $i rm -rf
/prod/HDOP/data/13/daas.na/yarn/nm/usercache/*; done

验证所有节点中的所有用户缓存目录是否为空。

启动YARN服务。

$ sudo service hadoop-mapreduce-historyserver start
$ sudo service hadoop-yarn-resourcemanager start
$ sudo service hadoop-yarn-nodemanager start

发送消息＆＃34; YARN缓存已被刷新。＆＃34;到我们的小组ID。

Answer 1

您可以使用此Ansible剧本来执行您的任务。

停止YARN服务。

- name: service hadoop-mapreduce-historyserver stop
  service: 
    name: hadoop-mapreduce-historyserver
    state: stopped

- name: service hadoop-yarn-resourcemanager stop
  service: 
    name: hadoop-yarn-resourcemanager
    state: stopped

- name: service hadoop-yarn-nodemanager stop
  service: 
    name: hadoop-yarn-nodemanager
    state: stopped

登录所有节点并删除用户缓存目录的内容。

- name: delete the content of the usercache directories
   file: path="{{ item }}" state=absent
   with_fileglob:
       - /prod/HDOP/data/13/daas.na/yarn/nm/usercache/*

验证所有节点中的所有用户缓存目录是否为空。

  - name:  Verify all usercache directories are empty.
    shell: /usr/bin/ls ~/soniya/ | wc -l
    register: total_size

  - debug:
         msg: "Directory total size is {{ total_size.stdout_lines }}"
        when: total_size.rc == 0

启动YARN服务。

- name: service hadoop-mapreduce-historyserver start
  service: 
    name: hadoop-mapreduce-historyserver
    state: started

- name: service hadoop-yarn-resourcemanager start
  service: 
    name: hadoop-yarn-resourcemanager
    state: started

- name: service hadoop-yarn-nodemanager start
  service: 
    name: hadoop-yarn-nodemanager
    state: started

将按摩发送至＆＃34; YARN Cache已刷新＆＃34;到我们的小组ID

您可以使用邮件模块。

如何使用Ansible清除Hadoop YARN缓存？

1 个答案: