Logstash grok multiline message

Question

My logs are formatted like this:

2014-06-19 02:26:05,556 INFO ok
2014-06-19 02:27:05,556 ERROR
 message:space exception
         at line 85
 solution:increas         


        

Answer 1

It looks like you have two issues:

You need to correctly combine your multilines:

filter
{
    multiline
   {
        pattern => "^ "
        what => "previous"
   }
}

This will combine any line that begins with a space into the previous line. You may end up having to use a "next" instead of a "previous".

Replace Newlines

I don't believe that grok matches across newlines.

I got around this by doing the following in your filter section. This should go before the grok section:

mutate
{
    gsub => ["message", "\n", "LINE_BREAK"]
}

This allowed me to grok multilines as one big line rather than matching only till the "\n".

Answer 2

As for multiline grok, it's best to use special flag for pattern string:

grok {
    match => ["message", "(?m)%{SYSLOG5424LINE}"]
}