Question

我想检查日志映射器或reducer输出？我在syslog下的容器foler中找不到它？那么日志输出在哪里？

        public class SkipStat {
private static Log log = LogFactory.getLog(SkipStat.class);

private static BlockWorkerRepository blockWorkerRepository;
static {
    blockWorkerRepository = new BlockWorkerRepositoryImpl();
}

private static class SkipInfoMapper extends Mapper<Object, BSONObject, Text, AssignmentWritable> {

    private final String invalidResult = "^";

    private static final Calendar currentCalendar = Calendar.getInstance();
    static {
        currentCalendar.add(Calendar.HOUR, -24);
    }

    protected void map(Object key, BSONObject value, Context context) throws IOException, InterruptedException {
        String result = (String) value.get("result");
        log.info("lol... get one result " + result); // LOG ...
        if (invalidResult.equals(result)) {

Answer 1

启用日志聚合，然后允许您运行＆＃34;纱线日志＆＃34;命令查看应用程序ID的日志，在yarn-site.xml中设置这些：

yarn.log-aggregation-enable
yarn.log-aggregation.retain-seconds
yarn.log-aggregation.retain-check-interval-seconds

进行此更改后似乎需要重新启动纱线。

请注意，在启用聚合后，注释中引用的命令将起作用：

yarn logs -applicationId application_1397033985470_0006

请参阅：http://hadoop.apache.org/docs/r2.2.0/hadoop-yarn/hadoop-yarn-common/yarn-default.xml

2.3.0文档中提供了这些参数的说明：http://hadoop.apache.org/docs/r2.3.0/hadoop-yarn/hadoop-yarn-common/yarn-default.xml

Answer 2

历史服务器正在收集这些日志并保留它们。您可以使用historyserver WebUI或使用yarn logs CLI查看它们。见Simplifying user-logs management and access in YARN。在他们上传日志之前是：

记录属于单个应用程序的所有容器及其在给定的NM上运行会聚合并写出一个（可能是压缩的）日志文件在FS的配置位置

ApplicationMasterUI将显示当前正在执行的应用程序日志。

yarn [hadoop 2.2]位置映射器或reducer log outputed在哪里？

2 个答案: