非交互式bash shell的环境变量

时间:2017-09-07 13:16:22

标签: java linux bash shell unix

我们有一个非交互式bash shell命令,它执行一些代码并且该代码读取一些环境变量。

 #!/bin/bash 
  export PATH="/usr/sbin:/sbin:/usr/lib/ambari-server/*:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/var/lib/ambari-agent"
    export HADOOP_CONF_DIR=${HADOOP_CONF_DIR:-"/usr/hdp/current/hadoop-client/conf"}
    export JAVA_HOME=${JAVA_HOME:-"/usr/java/default"}
    export LANG="en_US.UTF-8"
    export NM_HOST="test.com"
    export LOGNAME="root"
    export JVM_PID="$$"
    export PWD="/opt/hadoop/yarn/local/usercache/root/appcache/application_1504729685201_0003/container_e11_1504729685201_0003_01_000002"
    export LOCAL_DIRS="/opt/hadoop/yarn/local/usercache/root/appcache/application_1504729685201_0003"
    export SHDP_CONTAINERID="container_e11_1504729685201_0003_01_000002"
    export NM_HTTP_PORT="8042"
    export LOG_DIRS="/opt/hadoop/yarn/log/application_1504729685201_0003/container_e11_1504729685201_0003_01_000002"
    export NM_AUX_SERVICE_mapreduce_shuffle="AAA0+gAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=
    "
    export NM_PORT="45454"
    export USER="root"
    export HADOOP_YARN_HOME=${HADOOP_YARN_HOME:-"/usr/hdp/current/hadoop-yarn-nodemanager"}
    export CLASSPATH="/org/test/import/application.yml:./*"
    export HADOOP_TOKEN_FILE_LOCATION="/opt/hadoop/yarn/local/usercache/root/appcache/application_1504729685201_0003/container_e11_1504729685201_0003_01_000002/container_tokens"
    export NM_AUX_SERVICE_spark_shuffle=""
    export LOCAL_USER_DIRS="/opt/hadoop/yarn/local/usercache/root/"
    export HADOOP_HOME="/usr/hdp/2.6.1.0-129/hadoop"
    export HOME="/home/"
    export NM_AUX_SERVICE_spark2_shuffle=""
    export CONTAINER_ID="container_e11_1504729685201_0003_01_000002"
    export MALLOC_ARENA_MAX="4"
    ln -sf "/opt/hadoop/yarn/local/usercache/root/appcache/application_1504729685201_0003/filecache/11/application.yml" "application.yml"
    hadoop_shell_errorcode=$?
    if [ $hadoop_shell_errorcode -ne 0 ]
    then
      exit $hadoop_shell_errorcode
    fi
    ln -sf "/opt/hadoop/yarn/local/usercache/root/appcache/application_1504729685201_0003/filecache/12/test.jar" "test.jar"
    hadoop_shell_errorcode=$?
    if [ $hadoop_shell_errorcode -ne 0 ]
    then
      exit $hadoop_shell_errorcode
    fi
    # Creating copy of launch script
    cp "launch_container.sh" "/opt/hadoop/yarn/log/application_1504729685201_0003/container_e11_1504729685201_0003_01_000002/launch_container.sh"
    chmod 640 "/opt/hadoop/yarn/log/application_1504729685201_0003/container_e11_1504729685201_0003_01_000002/launch_container.sh"
    # Determining directory contents
    echo "ls -l:" 1>"/opt/hadoop/yarn/log/application_1504729685201_0003/container_e11_1504729685201_0003_01_000002/directory.info"
    ls -l 1>>"/opt/hadoop/yarn/log/application_1504729685201_0003/container_e11_1504729685201_0003_01_000002/directory.info"
    echo "find -L . -maxdepth 5 -ls:" 1>>"/opt/hadoop/yarn/log/application_1504729685201_0003/container_e11_1504729685201_0003_01_000002/directory.info"
    find -L . -maxdepth 5 -ls 1>>"/opt/hadoop/yarn/log/application_1504729685201_0003/container_e11_1504729685201_0003_01_000002/directory.info"
    echo "broken symlinks(find -L . -maxdepth 5 -type l -ls):" 1>>"/opt/hadoop/yarn/log/application_1504729685201_0003/container_e11_1504729685201_0003_01_000002/directory.info"
    find -L . -maxdepth 5 -type l -ls 1>>"/opt/hadoop/yarn/log/application_1504729685201_0003/container_e11_1504729685201_0003_01_000002/directory.info"

    exec /bin/bash -c "$JAVA_HOME/bin/java -jar test.jar  1>/opt/hadoop/yarn/log/application_1504729685201_0003/container_e11_1504729685201_0003_01_000002/Container.stdout 2>/opt/hadoop/yarn/log/application_1504729685201_0003/container_e11_1504729685201_0003_01_000002/Container.stderr"
    hadoop_shell_errorcode=$?
    if [ $hadoop_shell_errorcode -ne 0 ]
    then
      exit $hadoop_shell_errorcode

我们在〜/ .bashrc文件中添加了代码所需的环境变量,但似乎非交互式shell不读取此文件,因此我们的代码无法获取所需的env变量。不幸的是我正在使用的bash脚本是自动生成的,我无法在那里导出我的env变量。有没有其他方法可以使env变量可用于我们的Java代码?感谢。

1 个答案:

答案 0 :(得分:0)

可以在/etc/hadoop/conf/yarn-site.xml中配置特别需要的环境变量,那里有一个属性,yarn.nodemanager.admin-env,用于列出应该从中转发的环境变量NodeManager的环境到容器的。 默认值为MALLOC_ARENA_MAX = $ MALLOC_ARENA_MAX,您只需要在逗号分隔列表中附加需要传递的变量: 例如 MALLOC_ARENA_MAX = $ MALLOC_ARENA_MAX,YOUR_VARIABLE = $ YOUR_VARIABLE

这也可以在纱线的纱线现场高级配置部分使用Ambari进行配置。

使用/etc/profile.d/中的脚本设置全局环境变量是一种很好的做法,可供所有用户使用。

参考文献: - 纱线网站默认配置 https://hadoop.apache.org/docs/r0.23.11/hadoop-yarn/hadoop-yarn-common/yarn-default.xml