I am using Logstash and Elasticsearch versions 5.6.5. So far used elasticsearch output with HTTP protocol and no authentication. Now Elasticsearch is being secured using basic authentication (user/password) and CA certified HTTPS URL. I don't have any control over the elasticsearch server. I just use it to output to from Logstash.
Now when I try to configure the HTTPS URL of elasticsearch with basic authentication, it fails to create the pipeline.
Output Configuration
output {
elasticsearch {
hosts => ["https://myeslasticsearch.server.io"]
user => "esusername"
password => "espassword"
ssl => true
}
}
Errors
1. Error registering plugin {:plugin=>"#<LogStash::OutputDelegator:0x50aa9200
2. Pipeline aborted due to error {:exception=>#<URI::InvalidComponentError: bad component(expected user component):
How to fix this?
I notice that there is a field called cacert
which requires some PEM file. But I am not sure what to put there since the Elasticsearch server is using a CA certified SSL not a self-signed one.
Addtional question: I don't have any xpack installed. Is 'xpack' required to be purchased for HTTPS output to Elasticsearch from Logstash?
I found the root cause of the issue. There were three things to fix:
The logstash version I tested with was wrong 5.5.0. I downloaded the correct version to match with Elasticsearch Version 5.6.5.
The host I used was running on 443 port. When I didn't specify the port as below logstash appended 9200 with it, due to which the connection failed.
hosts => ['https://my.es.server.com']
Below configuration corrected the port used by logstash.
hosts => ['https://my.es.server.com:443']
I was missing proxy connection settings.
proxy => 'http://my.proxy.com:80'
Overall settings that worked.
output {
elasticsearch {
hosts => ['https://my.es.server.com:443']
user => 'esusername'
password => 'espassword'
proxy => 'http://my.proxy:80'
index => "my-index-%{+YYYY.MM.dd}"
}
}
No need for 'ssl' field.
Also NO need for 'xpack' installation for this requirement.
来源:https://stackoverflow.com/questions/49109251/how-to-authenticate-logstash-output-to-a-secure-elasticsearch-url-version-5-6-5