Question

 [25-Dec-2015 08:06:45] 0:: users to chek for delete
 [25-Dec-2015 08:08:44] 0:: users to chek for delete
 [25-Dec-2015 08:10:44] 3:: users to chek for delete
 [25-Dec-2015 08:10:44] Expected response code 200, got 404

     {
         "error": {
          "errors": [
           {
            "domain": "global",
            "reason": "notFound",
            "message": "Resource Not Found: userKey"
           }
          ],
          "code": 404,
          "message": "Resource Not Found: userKey"
         }
        }

    [06-Nov-2015 19:24:19 GMT] PHP Fatal error:  Class 'Test\Test\Api\Resources\Authenticate1234' not found in /apps/test/src/Test/Test/Api/Resources/ResourceFactory.php on line 10
    [06-Nov-2015 19:24:19 GMT] PHP Stack trace:
    [06-Nov-2015 19:24:19 GMT] PHP   1. {main}() /apps/test/public/api.php:0
    [06-Nov-2015 19:24:19 GMT] PHP   2. Test\Test\Api\ApiController->handleRequest() /apps/test/public/api.php:13
    [06-Nov-2015 19:24:19 GMT] PHP   3. Test\Test\Api\Resources\ResourceFactory->create() /apps/test/src/Test/Test/Api/ApiController.php:14

以上是我的日志文件示例。我需要过滤掉每条消息。问题在于编写过滤器。前3行是三个不同的错误。

[25-Dec-2015 08:06:45] 0 ::用户要删除
[25-Dec-2015 08:06:45] 0 ::用户要删除
[2015年12月25日08:06:45] 3 ::用户要删除

第四个错误是JSON消息出错。我需要将此块与上面分开。

    [25-Dec-2015 08:10:44] Expected response code 200, got 404
     {
         "error": {
          "errors": [
           {
            "domain": "global",
            "reason": "notFound",
            "message": "Resource Not Found: userKey"
           }
          ],
          "code": 404,
          "message": "Resource Not Found: userKey"
         }
        }

第五个错误是PHP堆栈跟踪。

        [06-Nov-2015 19:24:19 GMT] PHP Fatal error:  Class 'Test\Test\Api\Resources\Authenticate1234' not found in /apps/test/src/Test/Test/Api/Resources/ResourceFactory.php on line 10
        [06-Nov-2015 19:24:19 GMT] PHP Stack trace:
        [06-Nov-2015 19:24:19 GMT] PHP   1. {main}() /apps/test/public/api.php:0
        [06-Nov-2015 19:24:19 GMT] PHP   2. Test\Test\Api\ApiController->handleRequest() /apps/test/public/api.php:13
        [06-Nov-2015 19:24:19 GMT] PHP   3. Test\Test\Api\Resources\ResourceFactory->create() /apps/test/src/Test/Test/Api/ApiController.php:14

是否可以设计一个grok过滤器来匹配这三个条件？

Answer 1

使用multiline选项。例如：

filter {
    multiline {
        negate    => true
        pattern   => "^\["
        what      => "previous"
    }
}

结果应如下所示：

[06-Nov-2015 19:24:19 GMT] PHP Fatal error:  Class 'Test\Test\Api\Resources\Authenticate1234' not found in /apps/test/src/Test/Test/Api/Resources/ResourceFactory.php on line 10
PHP Stack trace:
PHP   1. {main}() /apps/test/public/api.php:0
PHP   2. Test\Test\Api\ApiController->handleRequest() /apps/test/public/api.php:13
PHP   3. Test\Test\Api\Resources\ResourceFactory->create() /apps/test/src/Test/Test/Api/ApiController.php:14

Answer 2

您的第一步是将多行json错误放入一个logstash事件中。查看多行编解码器或过滤器。然后，我建议使用一个grok {}节将日期时间拉出线，然后使用另一个grok节来处理该行的剩余部分。

如何使用grok对logstash中的日志消息进行分组？

2 个答案: