如何访问第一次尝试的纱线日志?

时间:2017-02-07 20:19:54

标签: logging yarn

如果我使用 _ 尝试 postfix,我会获得给定尝试的日志吗?像这样:

yarn logs -applicationId application_11112222333333_444444_1

奇怪的是,我没有在网上找到答案。

更新: 让我重新解释一下我的问题: 如何访问给定尝试的纱线日志?

1 个答案:

答案 0 :(得分:3)

这里有点丑陋但有效的解决方案分几步(对于hadoop-2.6)。基本上每次尝试都在它的容器中执行。要获取特定容器的日志,需要知道applicationId,containerId和节点管理器地址。例如,您需要获取appattempt_1的日志:

  1. 获取有关appattempt(containerId,host url)的信息: yarn applicationattempt -list application_ID_1。你会得到这样的东西:
  2. ======================== ======== ==================== =========================== 
      ApplicationAttempt-Id    State    AM-Container-Id            Tracking-URL         
     ======================== ======== ==================== =========================== 
      appattempt_1             FAILED   container_1          https://host1:8090/blabla  
      appattempt_2             KILLED   container_2          https://host2:8090/blabla  
     ======================== ======== ==================== =========================== 
    
    1. 要将tracking-URL转换为节点地址: $ yarn node -list -all | grep host1 | awk '{print $1}' host1:8041

    2. yarn logs -applicationId application_ID_1 -containerId container_1 -nodeAddress host1:8041

    3. 在hadoop-2.7中你可以使用:

      yarn logs -applicationId  [OPTIONS]
      
      general options are:
       -am                      Prints the AM Container logs for
                                               this application. Specify
                                               comma-separated value to get logs
                                               for related AM Container. For
                                               example, If we specify -am 1,2,
                                               we will get the logs for the
                                               first AM Container as well as the
                                               second AM Container. To get logs
                                               for all AM Containers, use -am
                                               ALL. To get logs for the latest
                                               AM Container, use -am -1. By
                                               default, it will print all
                                               available logs. Work with
                                               -log_files to get only specific
                                               logs.