logstash-configuration

Currently I am working on an application where I need to create documents from particular data from a file at specific location. I have set up logstash pipeline configuration. Here is what it looks like currently: input{ file{ path => "D:\ELK_Info\logstashInput.log" start_position => "beginning" } } #Possible IF condition here in the filter output { #Possible IF condition here http { url => "http://localhost:9200/<index_name>/<type_name>" http_method => "post" format => "json" } } I want to provide IF condition in output before calling API. The condition should be like, "If data from input

input { jdbc { jdbc_driver_library => "sqljdbc4.jar" jdbc_driver_class => "com.microsoft.sqlserver.jdbc.SQLServerDriver" jdbc_connection_string => "jdbc:sqlserver://192.168.2.126\\SQLEXPRESS2014:1433;databaseName=test jdbc_password => "sa@sa2015" schedule => "0 0-59 0-23 * * *" statement => "SELECT ID , Name, City, State,ShopName FROM dbo.Shops" jdbc_paging_enabled => "true" jdbc_page_size => "50000" } } filter { } output { stdout { codec => rubydebug } elasticsearch { protocol => "http" index => "shops" document_id => "%{id}" } } I want to import data in ElasticSearch using Logstash using

I am having trouble using sprintf to reference the event fields in the elasticsearch output plugin and I'm not sure why. Below is the event received from Filebeat and sent to Elasticsearch after filtering is complete: { "beat" => { "hostname" => "ca86fed16953", "name" => "ca86fed16953", "version" => "6.5.1" }, "@timestamp" => 2018-12-02T05:13:21.879Z, "host" => { "name" => "ca86fed16953" }, "tags" => [ [0] "beats_input_codec_plain_applied", [1] "_grokparsefailure" ], "fields" => { "env" => "DEV" }, "source" => "/usr/share/filebeat/dockerlogs/logstash_DEV.log", "@version" => "1", "prospector" =

I have two URLs (due to security concern i will explain by using dummy) a> https://xyz.company.com/ui/api/token b> https://xyz.company.com/request/transaction?date=2016-01-21&token=<tokeninfo> When you hit url mentioned in point 'a' it will generate a token let it be a string of 16 characters Then that token should be used in making second request of point 'b' in token param Updated The second url response is important to me i.e is a JSON response, I need to filter the json data and extract required data and output it to standard output and elastic search. is there any way of doing so in

I am using logstash jdbc to keep the things syncd between mysql and elasticsearch. Its working fine for one table. But now I want to do it for multiple tables. Do I need to open multiple in terminal logstash agent -f /Users/logstash/logstash-jdbc.conf each with a select query or do we have a better way of doing it so we can have multiple tables being updated. my config file input { jdbc { jdbc_driver_library => "/Users/logstash/mysql-connector-java-5.1.39-bin.jar" jdbc_driver_class => "com.mysql.jdbc.Driver" jdbc_connection_string => "jdbc:mysql://localhost:3306/database_name" jdbc_user =>