Question

我正在尝试为使用Log4J创建的Spark日志创建Java解析器。我写了这段代码来识别一个起始任务日志行，但它不起作用，我无法弄清楚原因。

这是正则表达式：

public static final String datePattern = "\\d{4}\\-\\d{2}\\-\\d{2}";
public static final String timePattern = "\\d{2}\\:\\d{2}\\:\\d{2}\\,\\d{3}";
public static final String timeStampPattern = "(?<timeStamp>" + datePattern + "\\s" + timePattern + ")";
public static final String logLevelPattern = "(?<logLevel>\\w+)";
public static final String loggingClassPattern = "(?<loggingClass>\\w+:)";
public static final String taskUIdPattern = "(?<UIdPattern>\\d+)";
public static final String taskIdPattern = "\\d.\\d:\\d+";
public static final String taskStatusPattern = null;
public static final String endTaskLabelPattern = null;
public static final String stringPatternStartTask = timeStampPattern + 
        " " + logLevelPattern + 
        " " + loggingClassPattern + 
        " " + "Starting task" +
        " " + taskIdPattern +
        " " + "as TID" +
        " " + taskUIdPattern +
        "\\z";

这是解析尝试：

Pattern patternStartTask = Pattern.compile(stringPatternStartTask);
...
while((temp = br.readLine()) != null) {
if((m = patternStartTask.matcher(temp)).matches()) {
    System.out.println(temp);
    le = new StartTaskEvent();
}
...
if(m != null && le != null) {
    le.setTaskId(m.group("taskId"));
    le.setLogLevel(m.group("logLevel"));
    le.setLoggingClass(m.group("loggingClass"));
    le.setTimeStamp(sdf.parse(m.group("timeStamp")));
    result.add(le);
}
}

我试图识别的线条就像这样：

2016-01-08 14:01:02 INFO TaskSetManager: Starting task 1.0:0 as TID 0 on executor 1

Answer 1

你的正则表达式结束于：

    " " + "as TID" +
    " " + taskUIdPattern +
    "\\z";

但是在你的字符串中，on executor 1之后taskUIdPattern，on executor 1之后你必须在你的正则表达式中添加on executor \\d或更好，taskUIdPattern

使用正则表达式的Java的火花日志解析器

1 个答案: