Question

有一个这样的日志文件，里面有很多条目：

2016-09-16 15:18:52,555 DEBUG Thread-3
2016-09-16 15:18:52,555 DEBUG Thread-3
2016-09-16 15:18:52,555 DEBUG Thread-3
2016-09-16 15:18:52,555 DEBUG Thread-3
2016-09-16 15:18:52,555 DEBUG Thread-3

Report Job information
Job ID : 12345
Job name : Background Execution
Job priority : 5
Job group : normal
Report group : General
Job description : Brief
User : user01
Report : General101
Modified user : 12
Modified date : 2016-09-16 15:18:52.08
Report mode : Simple

2016-09-16 15:18:52,555 DEBUG Thread-3
2016-09-16 15:18:52,555 DEBUG Thread-3
2016-09-16 15:18:52,555 DEBUG Thread-3
2016-09-16 15:18:52,555 DEBUG Thread-3
2016-09-16 15:18:52,555 DEBUG Thread-3

Report Job information
Job ID : 12367
Job name : Background Execution
Job priority : 5
Job group : normal
Report group : General1
Job description : Brief
User : user01
Report : General101
Modified user : 12
Modified date : 2016-09-16 15:45:52.08
Report mode : Simple

我需要能够从日志中提取此部分并放入数据框格式。日志将包含许多这样的条目，每次脚本在日志中看到此设置条目时，它都应附加到数据框中。

    Report Job information
    Job ID : 12367
    Job name : Background Execution
    Job priority : 5
    Job group : normal
    Report group : General1
    Job description : Brief
    User : user01
    Report : General101
    Modified user : 12
    Modified date : 2016-09-16 15:45:52.08
    Report mode : Simple

我试过这个：

  i<-c("app.log")
    xx <- readLines(i)
    ii <- 10 + grep("Report Job information",xx, fixed=TRUE)[1]
    line_count <- Filter(function(v) v!="", strsplit(xx[ii]," ")[[2]])

任何想法怎么能这样做？

鉴于此数据：

Job ID : 12345
Job name : Background Execution
Job priority : 5
Job group : normal
Report group : General
Job description : Brief
User : user01
Report : General101
Modified user : 12
Modified date : 2016-09-16 15:18:52.08
Report mode : Simple


Each set of entry need to be one row. For example:
title=Report Job information,jobid=12345, JobName=Background Execution, job priority=5,jobgroup=normal, reportgroup=General,ModifiedDate=2016-09-16 15:18:52.08 etc

Answer 1

以下是数据，读入（注意，使用dplyr / magrittr管道）：

inData <-
"2016-09-16 15:18:52,555 DEBUG Thread-3
2016-09-16 15:18:52,555 DEBUG Thread-3
2016-09-16 15:18:52,555 DEBUG Thread-3
2016-09-16 15:18:52,555 DEBUG Thread-3
2016-09-16 15:18:52,555 DEBUG Thread-3

Report Job information
Job ID : 12345
Job name : Background Execution
Job priority : 5
Job group : normal
Report group : General
Job description : Brief
User : user01
Report : General101
Modified user : 12
Modified date : 2016-09-16 15:18:52.08
Report mode : Simple

2016-09-16 15:18:52,555 DEBUG Thread-3
2016-09-16 15:18:52,555 DEBUG Thread-3
2016-09-16 15:18:52,555 DEBUG Thread-3
2016-09-16 15:18:52,555 DEBUG Thread-3
2016-09-16 15:18:52,555 DEBUG Thread-3

Report Job information
Job ID : 12367
Job name : Background Execution
Job priority : 5
Job group : normal
Report group : General1
Job description : Brief
User : user01
Report : General101
Modified user : 12
Modified date : 2016-09-16 15:45:52.08
Report mode : Simple" %>%
  strsplit("\\n") %>%
  unlist

我认为最简单的方法是先找到所有的起始位置。在这里，我假设＆＃34;报告工作信息＆＃34;在每个报告的顶部（但它不需要包含在data.frame中，所以我在它之后开始一个）。

reportStarts <-
  grep("Report Job information", inData) + 1

接下来，我找到报告的结束位置。我不太了解结构，我假设＆＃34;报告模式＆＃34;是报道的最后一件事。相反，如果是设定的行数或其他一些中断指标，则可能需要修改此步骤。

reportStops <-
  grep("Report mode", inData)

接下来，我遍历每个报告（使用idx来指示我正在处理哪个报告）。该方法首先限制本报告中的行，然后将每个行分为＆＃34; ：＆＃34;它似乎是现场指示器，并使用＆＃34;之前的部分。：＆＃34;命名输出（我使用粘贴作为防御，以防值还包括＆＃34;：＆＃34;）。最后，我将每个都拼接成一个data.frame，然后用bind_rows绑定行，以确保它适当地处理某些报告中可能存在的任何额外列。

lapply(1:length(reportStarts), function(idx){

  inData[reportStarts[idx]:reportStops[idx]] %>%
    strsplit(" : ") %>%
    sapply(function(thisField){
      setNames(paste(thisField[-1]
                     , collapse = " : ")
               , thisField[1])
    }) %>%
    rbind %>%
    data.frame()

}) %>%
  bind_rows()

输出此data.frame：

  Job.ID             Job.name Job.priority Job.group Report.group Job.description   User     Report Modified.user          Modified.date Report.mode
1  12345 Background Execution            5    normal      General           Brief user01 General101            12 2016-09-16 15:18:52.08      Simple
2  12367 Background Execution            5    normal     General1           Brief user01 General101            12 2016-09-16 15:45:52.08      Simple

Answer 2

您可以尝试：

library(stringr)

foo <- function(x, y){
  tmp <- y[c(x:(x+11))] # extract
  tmp <- str_split(tmp, ": ") # split and optimize
  unlist(lapply(tmp, tail, 1)) # unlist and output last element
  }

data.frame(t(cbind(sapply(i, foo, xx))))
                      X1    X2                   X3 X4     X5       X6    X7     X8         X9 X10
1 Report Job information 12345 Background Execution  5 normal  General Brief user01 General101  12
2 Report Job information 12367 Background Execution  5 normal General1 Brief user01 General101  12
                     X11    X12
1 2016-09-16 15:18:52.08 Simple
2 2016-09-16 15:45:52.08 Simple

如何从R日志文件中选择某些行

2 个答案: