logstash-configuration

问题 Presently I have my logs and logstash running on the same machine, so I read my logs placed on my local machine with this config(using pull model) input { file { path => "/home/Desktop/Logstash-Input/**/*_log" start_position => "beginning" } } Now, we have logstash running on a different machine and want to read the logs remote mechine. Is there a way to set the ip in file input of config file? EDIT: I manage to do this with logstash-forwarder which is a push model(log shipper/logstash

问题 Presently I have my logs and logstash running on the same machine, so I read my logs placed on my local machine with this config(using pull model) input { file { path => "/home/Desktop/Logstash-Input/**/*_log" start_position => "beginning" } } Now, we have logstash running on a different machine and want to read the logs remote mechine. Is there a way to set the ip in file input of config file? EDIT: I manage to do this with logstash-forwarder which is a push model(log shipper/logstash

问题 I'm trying to connect logstash (Version 1.5.0) to get logs of services (that run on apache-tomcat). These logs are log4j. I use this config for logstash: input { log4j { mode => server host => localhost port => 4560 type => "log4j" } }... and in my service' log4j.xml I've set my SocketAppender: <appender name="OHADS" class="org.apache.log4j.net.SocketAppender"> <param name="port" value="4560" /> <param name="remoteHost" value="localhost" /> </appender> It works fine. The questions: I want

问题 I have a large amount (~40000) of nested JSON objects I want to insert into elasticsearch an index. The JSON objects are structured like this: { "customerid": "10932" "date": "16.08.2006", "bez": "xyz", "birthdate": "21.05.1990", "clientid": "2", "address": [ { "addressid": "1", "tile": "Mr", "street": "main str", "valid_to": "21.05.1990", "valid_from": "21.05.1990", }, { "addressid": "2", "title": "Mr", "street": "melrose place", "valid_to": "21.05.1990", "valid_from": "21.05.1990", } ] } So

问题 I am using Logstash to parse a log file. A sample log line is shown below. 2011/08/10 09:51:34.450457,1.048908,tcp,213.200.244.217,47908, ->,147.32.84.59,6881,S_RA,0,0,4,244,124,flow=Background-Established-cmpgw-CVUT I am using following filter in my confguration file. grok { match => ["message","%{DATESTAMP:timestamp},%{BASE16FLOAT:value},%{WORD:protocol},%{IP:ip},%{NUMBER:port},%{GREEDYDATA:direction},%{IP:ip2},%{NUMBER:port2},%{WORD:status},%{NUMBER:port3},%{NUMBER:port4},%{NUMBER:port5},%

问题 I have set up Elk stack on my windows machine with the following : Elasticserach Logstash Kibana My logstash.conf input { file { path => "\bin\MylogFile.log" start_position => "beginning" } } output { elasticsearch { hosts => localhost:9200 } } MylogFile.log(Apache Log) 127.0.0.1 - frank [10/Oct/2000:13:55:36 -0700] "GET /apache_pb.gif HTTP/1.0" 200 2326 "http://www.example.com/start.html" "Mozilla/4.08 [en] (Win98; I ;Nav)" When I run logstash.conf it creates the following index in

问题 Error: c: \ Program Files \ Logstash \ bin> logstash.bat -e 'input {stdin {}} output {stdout {}}' An unexpected error occurred! : Error => bad URI (is not URI?): File: // c: / Program Files / Logstash / confi g / log4j2.properties,: backtrace => [ "C: / Program Files / Logstash / vendor / jruby / lib / ruby ​​/ 1.9 / uri / common. rb: 176: in split '", "C: / Program Files / Logstash / vendor / jruby / lib / ruby ​​/ 1.9 / uri / common.rb: 210: in parse ' "," C: / Program Files / Logstash /

问题 Error: c: \ Program Files \ Logstash \ bin> logstash.bat -e 'input {stdin {}} output {stdout {}}' An unexpected error occurred! : Error => bad URI (is not URI?): File: // c: / Program Files / Logstash / confi g / log4j2.properties,: backtrace => [ "C: / Program Files / Logstash / vendor / jruby / lib / ruby ​​/ 1.9 / uri / common. rb: 176: in split '", "C: / Program Files / Logstash / vendor / jruby / lib / ruby ​​/ 1.9 / uri / common.rb: 210: in parse ' "," C: / Program Files / Logstash /

问题 When using Logstash and Elasticsearch together, fields with .raw are appended for analyzed fields, so that when querying Elasticsearch with tools like Kibana, it's possible to use the field's value as-is without per-word splitting and what not. I built a new installation of the ELK stack with the latest greatest versions of everything, and noticed my .raw fields are no longer being created as they were on older versions of the stack. There are a lot of folks posting solutions of creating

问题 I have Nginx as loadbalancer which is generating logs without year and second information in timestamp. One of those logs are 08-10 09:28 root ERROR Error connecting to CRS REST API : [Errno 111] Connection refused Error connecting to CRS REST API : [Errno 111] Connection refused The pattern for this is : (?m)%{MONTHNUM:monthNum}\-%{MONTHDAY:monthDay}\s*%{HOUR:hour}:%{MINUTE:minute}\s*%{WORD}\s*%{LOGLEVEL_CUSTOM:severity}\s*%{GREEDYDATA:messagePayload} While I understand that year information