Question

我需要解析我的生产rails应用程序日志文件。我只需提取未经授权的请求。

这些部分以Started POST "/...some_url..."开头，以Completed 401 Unauthorized in {some_number}ms

结尾

我成功地使用cat mylog.log | grep -B 5 "Completed 401 Unauthorized"来扩展结尾，但是此命令在上下文之前仅捕获5个字符串。这种情况可能在很大的范围内变化。

那么，如何捕获起始字符串并获取整个Completed 401 Unauthorized上下文？

UPD：日志文件示例：

Started POST "/api/orders" for 111.222.333.444 at 2014-02-12 07:37:23 +0400
Processing by Api::V2::OrdersController#create as JSON
  Parameters: {bla
  bla
  bla}
WARNING: Can't verify CSRF token authenticity
  Partner Load (0.9ms)  SELECT `partners`.* FROM `partners` WHERE `partners`.`agent_referer` = 'abcde' LIMIT 1
Completed 401 Unauthorized in 4ms

...

Started GET "/search/flights?depart_city=SVX&destin_city=AER&depart_date=17.02.2014&return_date=21.02.2014" for 111.222.333.444 at 2014-02-12 06:29:19 +0400
Processing by Api::V1::TripsController#index as JSON
  Parameters: {bla bla bla}
  Partner Load (0.1ms)  SELECT `partners`.* FROM `partners` WHERE `partners`.`agent_referer` = 'xxx' LIMIT 1
  Partner Load (0.1ms)  SELECT `partners`.* FROM `partners` WHERE `partners`.`name` = 'xxx' LIMIT 1
  Partner Load (0.0ms)  SELECT `partners`.* FROM `partners` WHERE `partners`.`agent_referer` = 'xxx'
   (0.0ms)  SELECT COUNT(*) FROM `partner_statuses` WHERE `partner_statuses`.`partner_id` = 26 AND (active_to >= '2014-02-11 19:59:59')
  PartnerStatus Load (0.0ms) SQL REQUEST HERE
  Rendered api/v1/trips/index.xml.builder (0.6ms)
Completed 200 OK in 39ms (Views: 1.3ms | ActiveRecord: 25.7ms)

Answer 1

在awk中使用一个小状态机：

awk '
    $1 == "Started"   { capture = 1 } 
    capture           { lines[n++] = $0 } 
    $1 == "Completed" {
        capture=0
        if ($3 == "Unauthorized") 
            for (i=0; i<n; i++) 
                print lines[i]
        n=0
        delete lines
    }
' log.log

Ruby on Rails仅记录未经授权的请求进行解析

1 个答案: