多行日志的正则表达式

时间:2019-06-20 13:22:06

标签: regex ruby elasticsearch fluentd

我正在使用Fluentd将日志转发到AWS中的Elasticsearch Service。应用程序日志是这样的,我正在尝试使用multi_format解析器进行解析。当前,它被设置为将与regex不匹配的所有日志捕获到ES中的其他字段。

    <pattern>
      format regexp
      expression /(?<log_line>.*)/
    </pattern>

由于没有正则表达式匹配,因此以下日志位于该字段中。

"20-Jun-2019 11:38:20] WARNING: [pool www] child 109 said into stdout: "EXCPT: /var/www/core/Router.php:85  Route not found./helloadmin 
[20-Jun-2019 11:38:20] WARNING: [pool www] child 109 said into stdout: " #0 /var/www/index.php(37): admin\core\Router::parse()"
"20-Jun-2019 11:38:41] WARNING: [pool www] child 108 said into stdout: "EXCPT: /var/www/core/Router.php:85  Route not found./test1 
[20-Jun-2019 11:38:41] WARNING: [pool www] child 108 said into stdout: " #0 /var/www/index.php(37): admin\core\Router::parse()"
"20-Jun-2019 11:38:56] WARNING: [pool www] child 111 said into stdout: "EXCPT: /var/www/core/Router.php:85  Route not found./test2 
[20-Jun-2019 11:38:56] WARNING: [pool www] child 111 said into stdout: " #0 /var/www/index.php(37): admin\core\Router::parse()"
"20-Jun-2019 11:39:09] WARNING: [pool www] child 109 said into stdout: "EXCPT: /var/www/core/Router.php:85  Route not found./test3 
[20-Jun-2019 11:39:09] WARNING: [pool www] child 109 said into stdout: " #0 /var/www/index.php(37): admin\core\Router::parse()"
"20-Jun-2019 11:40:11] WARNING: [pool www] child 109 said into stdout: "EXCPT: /var/www/core/Router.php:85  Route not found./edfdf 
[20-Jun-2019 11:40:11] WARNING: [pool www] child 109 said into stdout: " #0 /var/www/index.php(37): admin\core\Router::parse()"
"20-Jun-2019 11:45:40] WARNING: [pool www] child 108 said into stdout: "EXCPT: /var/www/core/DB.php:59  No such file or directory 
[20-Jun-2019 11:45:40] WARNING: [pool www] child 108 said into stdout: " #0 /var/www/core/DB.php(45): admin\core\DB->connect()"
[20-Jun-2019 11:45:40] WARNING: [pool www] child 108 said into stdout: "#1 /var/www/core/Container.php(93): admin\core\DB->__construct(Array)"
[20-Jun-2019 11:45:40] WARNING: [pool www] child 108 said into stdout: "#2 /var/www/core/Container.php(115): admin\core\Container::admin\core\{closure}()"
[20-Jun-2019 11:45:40] WARNING: [pool www] child 108 said into stdout: "#3 /var/www/core/Util.php(1144): admin\core\Container::getService('dbWriter')"
[20-Jun-2019 11:45:40] WARNING: [pool www] child 108 said into stdout: "#4 /var/www/core/Auth.php(30): admin\core\Util::updateAdminLogOut()"
[20-Jun-2019 11:45:40] WARNING: [pool www] child 108 said into stdout: "#5 /var/www/controllers/Dashboard.php(43): admin\core\Auth::checkSession()"
[20-Jun-2019 11:45:40] WARNING: [pool www] child 108 said into stdout: "#6 /var/www/index.php(47): admin\controllers\Dashboard->index(Array)"
"20-Jun-2019 11:45:54] WARNING: [pool www] child 111 said into stdout: "EXCPT: /var/www/core/DB.php:59  No such file or directory 
[20-Jun-2019 11:45:54] WARNING: [pool www] child 111 said into stdout: " #0 /var/www/core/DB.php(45): admin\core\DB->connect()"
[20-Jun-2019 11:45:54] WARNING: [pool www] child 111 said into stdout: "#1 /var/www/core/Container.php(93): admin\core\DB->__construct(Array)"
[20-Jun-2019 11:45:54] WARNING: [pool www] child 111 said into stdout: "#2 /var/www/core/Container.php(115): admin\core\Container::admin\core\{closure}()"
[20-Jun-2019 11:45:54] WARNING: [pool www] child 111 said into stdout: "#3 /var/www/core/Util.php(1144): admin\core\Container::getService('dbWriter')"
[20-Jun-2019 11:45:54] WARNING: [pool www] child 111 said into stdout: "#4 /var/www/core/Auth.php(30): admin\core\Util::updateAdminLogOut()"
[20-Jun-2019 11:45:54] WARNING: [pool www] child 111 said into stdout: "#5 /var/www/controllers/Users.php(2194): admin\core\Auth::checkSession()"
[20-Jun-2019 11:45:54] WARNING: [pool www] child 111 said into stdout: "#6 /var/www/index.php(47): admin\controllers\Users->getUserSnapshot(Array)"

我正在寻找一种解析这些的方法。我尝试对多行使用/ m选项,并且没有它,但是一切都失败了。如果您不能给我确切的正则表达式,请给我一些想法来解析它。

我尝试使用子编号分隔日志-在这种情况下为109/108/111等,但是由于它们重叠,因此无法正确显示:

"20-Jun-2019 11:45:56] WARNING: [pool www] child 114 said into stdout: "EXCPT: /var/www/core/DB.php:59  No such file or directory
[20-Jun-2019 11:45:56] WARNING: [pool www] child 114 said into stdout: " #0 /var/www/core/DB.php(45): admin\core\DB->connect()"
[20-Jun-2019 11:45:56] WARNING: [pool www] child 114 said into stdout: "#1 /var/www/core/Container.php(93): admin\core\DB->__construct(Array)"
[20-Jun-2019 11:45:56] WARNING: [pool www] child 114 said into stdout: "#2 /var/www/core/Container.php(115): admin\core\Container::admin\core\{closure}()"
[20-Jun-2019 11:45:56] WARNING: [pool www] child 114 said into stdout: "#3 /var/www/core/Util.php(1144): admin\core\Container::getService('dbWriter')"
[20-Jun-2019 11:45:56] WARNING: [pool www] child 114 said into stdout: "#4 /var/www/core/Auth.php(30): admin\core\Util::updateAdminLogOut()"
[20-Jun-2019 11:45:56] WARNING: [pool www] child 114 said into stdout: "#5 /var/www/controllers/Dashboard.php(43): admin\core\Auth::checkSession()"
[20-Jun-2019 11:45:56] WARNING: [pool www] child 114 said into stdout: "#6 /var/www/index.php(47): admin\controllers\Dashboard->index(Array)"
"20-Jun-2019 11:46:13] WARNING: [pool www] child 114 said into stdout: "EXCPT: /var/www/core/Router.php:85  Route not found./test
[20-Jun-2019 11:46:13] WARNING: [pool www] child 114 said into stdout: " #0 /var/www/index.php(37): admin\core\Router::parse()"

子编号相同,但是这是两个不同的事件。请注意时间。

请有人帮忙。

0 个答案:

没有答案