Why are there no logstash indexes in kibana

问题

I set up ELK stack and filebeat with my ELK node as a RedHat server following the digitalocean tutorial. Kibana is up and running, but I dont see any logstash indexes when I go to configure an index pattern as logstash-*:

Unable to fetch mapping. Do you have any indices matching the pattern? 

When I do a curl to see the indexes I have, they are only filebeat indexes. Filebeat should be pushing data to logstash which is listening on 5044

$curl 'localhost:9200/_cat/indices?v'
health status index               pri rep docs.count docs.deleted store.size pri.store.size 
yellow open   filebeat-2017.01.10   5   1       3864            0      1.7mb          1.7mb 
yellow open   filebeat-2017.06.17   5   1       1848            0    740.1kb        740.1kb 
yellow open   filebeat-2017.01.18   5   1      77062            0       33mb           33mb 
yellow open   filebeat-2017.09.14   5   1       1932            0      1.1mb          1.1mb 
yellow open   filebeat-2017.01.11   5   1      19094            0      3.6mb          3.6mb 
yellow open   .kibana  

You can see I only have filebeat indexes. I checked my ports are open, and My config files are correct according to the tutorial. What could be wrong? Filebeat should be sending logs from /var/log/*.log to logstash, to elasticsearch.

When I

tail /var/log/logstash/logstash.log

there is nothing in my logstash log. I've checked and logstash, filebeat, kibana, and elasticsearch are all running. Ive also done the config file test and it said it was OK:

$sudo service logstash status
logstash is running

On my ELK node, I can clearly see the port 5044 is listening:

$ netstat -tulpn | grep -i listen | grep -v tcp6
 tcp        0      0 :::5044                     :::*                        LISTEN      -  

回答1:

Did you define your index in your Kibana, from Management > Index Patterns > Add New?

It's obvious that you won't be able to find the index which you've created using logstash in Kibana, unless you're manually creating it there within the Managemen section of Kibana.

Make sure, that you have the same name of the indice which you created using logstash. Have a look at the doc, which conveys:

When you define an index pattern, indices that match that pattern must exist in Elasticsearch. Those indices must contain data.

which pretty much says that the indice should exist for you to create the index in Kibana. What logstash does is, to only create the indices in Elasticsearch itself, where as you have to manually create them in Kibana in order to access the and visualize the data.

Hope it helps!

回答2:

Filebeat creates daily indices using a pattern of filebeat-YYYY.MM.dd so you should not expect to see logstash indices in Elasticsearch.

The Logstash configuration recommended in the Filebeat documentation writes the data to an index based on "%{[@metadata][beat]}-%{+YYYY.MM.dd}" where [@metadata][beat] defaults to the name of the beat (filebeat) unless output.logstash.index is configured in the Filebeat config. Here's the base configuration for Logstash.

input {
  beats {
    port => 5044
  }   
}   

output {
  elasticsearch {
    hosts => "localhost:9200"
    manage_template => false
    index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
    document_type => "%{[@metadata][type]}"
  }   
}

You can inspect the data in those indices to see if it's what you are expected to get from filebeat with a command like:

curl http://localhost:9200/filebeat-*/_search?pretty&size=100

来源：https://stackoverflow.com/questions/41722972/why-are-there-no-logstash-indexes-in-kibana