Telegraf tail with grok pattern error

旧时模样 提交于 2019-12-23 05:43:09

问题


I am using Telegraf to get logs information from Apache NiFi, for this task I am using this config:

[[inputs.tail]]
  ## files to tail.
  files = ["/var/log/nifi/nifi-app.log"]
  ## Read file from beginning.
  from_beginning = true
  #name_override = "nifi_app"

  ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
  data_format = "grok"
  grok_patterns = [ "%{DATE:date} %{TIME:time} %{WORD:EventType} \[%{GREEDYDATA:NifiTask} %{NOTSPACE:Thread}\] %{NOTSPACE:NifiEventType} %{GREEDYDATA:EventText} %{NUMBER:EventDuration} %{WORD:EventDurationUnits}" ]

When I try to start telegraf it give me this error:

Error parsing /etc/telegraf/telegraf.conf, toml: line 10: parse error

The pattern I wrote was tested in a Grok debugger with this text:

2018-08-02 10:53:16,976 INFO [Heartbeat Monitor Thread-1] o.a.n.c.c.h.AbstractHeartbeatMonitor Finished processing 1 heartbeats in 11863 nanos

These are the results of some testing:

grok_patterns = ["\[%{GREEDYDATA:NifiTask}\]"] ==> toml: line 10: parse error  
grok_patterns = ["[%{GREEDYDATA:NifiTask}]"] ==> Invalid data format: grok  
grok_patterns = ['\[%{GREEDYDATA:NifiTask}\]'] ==> Invalid data format: grok  
grok_patterns = ["\\[%{GREEDYDATA:NifiTask}\\]"] ==> Invalid data format: grok  
grok_patterns = ['[%{GREEDYDATA:NifiTask}]'] -> Invalid data format: grok  

The first option for me is the right one, but doesn't works, and the problem seems to be the way the bracket is being escaped.

How is possible to solve this issue?


回答1:


There was more than one problem:

  • First problem: the grok dataformat is added to Telegraf in the 1.8 release (ref), so I must use a nightly install until this version is released.

  • Second problem: how to escape the brackets, there are problems doing it in a regular way, so what I finally did was to put this part in a custom pattern file, this way it works perfectly.



来源:https://stackoverflow.com/questions/51655530/telegraf-tail-with-grok-pattern-error

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!