问题
[2018-02-12 09:15:43] development.WARNING: home page
[2018-02-12 09:15:43] development.INFO: home page
[2018-02-12 10:22:50] development.WARNING: home page
[2018-02-12 10:22:50] development.INFO: home page
[2018-02-12 10:22:50] development.ERROR: Call to undefined function vie() {"exception":"[object](Symfony\\Component\\Debug\\Exception\\FatalThrowableError(code: 0): Call to undefined function vie() at /var/www/html/routes/web.php:16
[stacktrace]
#0 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Route.php(198): Illuminate\\Routing\\Router->{closure}()
#1 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Route.php(172): Illuminate\\Routing\\Route->runCallable()
#2 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Router.php(658): Illuminate\\Routing\\Route->run()
#3 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php(30): Illuminate\\Routing\\Router->Illuminate\\Routing\\{closure}(Object(Illuminate\\Http\\Request))
#4 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Middleware/SubstituteBindings.php(41): Illuminate\\Routing\\Pipeline->Illuminate\\Routing\\{closure}(Object(Illuminate\\Http\\Request))
#5 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(149): Illuminate\\Routing\\Middleware\\SubstituteBindings->handle(Object(Illuminate\\Http\\Request), Object(Closure))
.....
.....
.....
#45 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(151): Illuminate\\Pipeline\\Pipeline->then(Object(Closure))
#46 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(116): Illuminate\\Foundation\\Http\\Kernel->sendRequestThroughRouter(Object(Illuminate\\Http\\Request))
#47 /var/www/html/public/index.php(55): Illuminate\\Foundation\\Http\\Kernel->handle(Object(Illuminate\\Http\\Request))
#48{main}
"}
Above is my Laravel monolog sample log data. I am using Logstash to read the log data and sent it to Elasticsearch. Below is my logstash.conf file
input {
file {
path => '/var/www/html/php-app/application/storage/logs/laravel-*.log'
start_position => 'beginning'
ignore_older => 0
}
}
filter {
grok {
match => { "message" => "\[%{TIMESTAMP_ISO8601:timestamp}\] %{DATA:env}\.%{DATA:severity}: %{DATA:message}at %{DATA:trace}" }
}
}
output {
elasticsearch {
hosts => [ 'localhost:9200' ]
index => "laravel-%{+YYYY-MM-dd}"
}
stdout {
codec => rubydebug
}
}
Above configuration is working for single line log messages. For example below log message
[2018-02-12 09:15:43] development.WARNING: home page
generates the output as
"timestamp": "2018-02-12 10:57:25",
"@timestamp": "2018-02-12T10:57:26.614Z",
"severity": "INFO",
"path": "/var/www/html/php-app/application/storage/logs/laravel-2018-02-12.log",
"message": "[2018-02-12 10:57:25] development.INFO: home page ",
"env": "development"
But for Multiline messages(i.e - message with stacktrace), it generates like below for each line.
"@timestamp" => 2018-02-12T10:56:47.785Z,
"path" => "/var/www/html/php-app/application/storage/logs/laravel-2018-02-12.log",
"message" => "#1 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(149): Illuminate\\\\Foundation\\\\Http\\\\Middleware\\\\ValidatePostSize->handle(Object(Illuminate\\\\Http\\\\Request), Object(Closure))",
"tags" => [
[0] "_grokparsefailure"
],
I have tried multi-line filter too. Still no success for multi-line error logs. I need a solution which suits for both multi-line and single line error messages.
Please help me in find right grok config which suits for both single line and multi-line error logs .
回答1:
Finally! I got the solution for my problem. and posting the logstash config which can be useful for others in future.
input {
file {
path => '/var/www/html/php-app/application/storage/logs/laravel-*.log'
start_position => 'beginning'
ignore_older => 0
codec => multiline { pattern => "\[[\d]{4}" negate => "true" what => "previous" }
}
}
filter {
grok {
match => { "message" => "\[%{TIMESTAMP_ISO8601:timestamp}\] %{DATA:env}\.%{DATA:severity}: %{DATA:message}" }
}
}
output {
elasticsearch {
hosts => [ 'localhost:9200' ]
index => "laravel-%{+YYYY-MM-dd}"
}
stdout {
codec => rubydebug
}
}
@baudsp Thanks for helping me out to solve this problem.
来源:https://stackoverflow.com/questions/48745506/logstash-grok-filter-config-for-php-monolog-multi-linestacktrace-logs