logstash-configuration

问题 I have a json in the form of [ { "foo":"bar" } ] I am trying to filter it using the json filter in logstash. But it doesn't seem to work. I found that I can't parse list json using the json filter in logstash. Can someone please tell me about any workaround for this? UPDATE My logs IP - - 0.000 0.000 [24/May/2015:06:51:13 +0000] *"POST /c.gif HTTP/1.1"* 200 4 * user_id=UserID&package_name=SomePackageName&model=Titanium+S202&country_code=in&android_id=AndroidID&eT=1432450271859&eTz=GMT%2B05

问题 Currently I am working on an application where I need to create documents from particular data from a file at specific location. I have set up logstash pipeline configuration. Here is what it looks like currently: input{ file{ path => "D:\ELK_Info\logstashInput.log" start_position => "beginning" } } #Possible IF condition here in the filter output { #Possible IF condition here http { url => "http://localhost:9200/<index_name>/<type_name>" http_method => "post" format => "json" } } I want to

问题 I have two URLs (due to security concern i will explain by using dummy) a> https://xyz.company.com/ui/api/token b> https://xyz.company.com/request/transaction?date=2016-01-21&token=<tokeninfo> When you hit url mentioned in point 'a' it will generate a token let it be a string of 16 characters Then that token should be used in making second request of point 'b' in token param Updated The second url response is important to me i.e is a JSON response, I need to filter the json data and extract

问题 I am using logstash jdbc to keep the things syncd between mysql and elasticsearch. Its working fine for one table. But now I want to do it for multiple tables. Do I need to open multiple in terminal logstash agent -f /Users/logstash/logstash-jdbc.conf each with a select query or do we have a better way of doing it so we can have multiple tables being updated. my config file input { jdbc { jdbc_driver_library => "/Users/logstash/mysql-connector-java-5.1.39-bin.jar" jdbc_driver_class => "com

问题 I have a Logstash configuration that I've been using to forward log messages in emails. It uses json and json_encode to parse and re-encode JSON log messages. json_encode used to pretty-print the JSON, which made for very nice looking emails. Unfortunately, with recent Logstash upgrades, it no longer pretty prints. Is there any way I can get a pretty form of the event into a field that I can use for the email bodies? I'm fine with JSON, Ruby debug, or most other human readable formats. filter

问题 here is my twitter input tweets "_source": { "created_at": "Wed Aug 10 06:42:48 +0000 2016", "id": 763264318242783200, "timestamp_ms": "1470811368891", "@version": "1", "@timestamp": "2016-08-10T06:42:48.000Z" } and my logstash config file which include twitter input plugin filter and output input { twitter { consumer_key => "lvvoeonCRBOHsLAoTPbion9sK" consumer_secret => "GNHOFzErJhuo0bNq38JUs7xea2BOktMiLa7tunoGwP0oFKCHrY" oauth_token => "704578110616936448-gfeSklNrITu7fHIZgjw3nwoZ1S0l0Jl"

问题 I need the common RE for representing the below values Invoice_IID: 00000000-4164-1638-e168-ffff08d24460 Invoice_IID 00000000-4164-1638-e168-ffff08d24460 invoice iid 00000000-4164-1638-074f-ffff08d24461 <invoice iid="00000000-4164-1638-074f-ffff08d24461" <invoice iid=\"00000000-4164-1638-074f-ffff08d24461\" <parent_invoice iid="00000000-4164-1638-074f-ffff08d24461" I am trying with the below configuration with my grok debugger like http://grokconstructor.appspot.com/do/match#result grok {

问题 Logstash.conf input { tcp { port => 7199 } } output { elasticsearch { hosts => ["localhost"] } } Cassandra running on 7199 port and jhipster application running on localhost:8080. we are unable to add into logstash by my_application No log4j2 file found. 回答1: I think you can use the JDBC plugin: https://github.com/logstash-plugins/logstash-input-jdbc input { jdbc { jdbc_connection_string => "jdbc:cassandra://hostname:XXXX" # Your port jdbc_user => "user" # The user value jdbc_password =>

问题 I had like to know if it is possible to send only specific log messages to elasticsearch via logstash? E.G let's say I have these messages in my log file: 2015-08-14 12:21:03 [31946] PASS 10.249.10.70 http://google.com 2015-08-14 12:25:00 [2492] domainlist \"/etc/ufdbguard/blacklists\ 2015-08-14 12:21:03 [31946] PASS 10.249.10.41 http://yahoo.com I had like to skip the second line when logstash/log forwarder process this log, is it possible to instruct it to skip any log message with the

问题 I am newbie to ELK and i'm writing a config file which uses multiline and we need to write a pattern for input data 110000|read|<soapenv:Envelope> <head>hello<head> <body></body> </soapenv:Envelope>|<soapenv:Envelope> <body></body> </soapenv:Envelope> 210000|read|<soapenv:Envelope> <head>hello<head> <body></body> </soapenv:Envelope>|<soapenv:Envelope> <body></body> </soapenv:Envelope> 370000|read|<soapenv:Envelope> <head>hello<head> <body></body> </soapenv:Envelope>|<soapenv:Envelope> <body><