How to stop logstash from creating a default mapping in ElasticSearch

Question

I am using logstash to feed logs into ElasticSearch. I am configuring logstash output as:

input {
file {
            path => \"/tmp/foo.log\"
            code         


        

Answer 1

You will need a mapping to store data in Elasticsearch and to search on it - that's how ES knows how to index and search those content types. You can either let logstash create it dynamically or you can prevent it from doing so and instead create it manually.

Keep in mind you cannot change existing mappings (although you can add to them). So first off you will need to delete the existing index. You would then modify your settings to prevent dynamic mapping creation. At the same time you will want to create your own mapping.

For example, this will create the mappings for the logstash data but also restrict any dynamic mapping creation via "strict":

$ curl -XPUT 'http://localhost:9200/4glogs/logs/_mapping' -d '
{
    "logs" : {
        "dynamic": "strict",
        "properties" : {
            "@timestamp": {
                "type": "date",
                "format": "dateOptionalTime"
                    },
            "@version": {
                "type": "string"
                    },
             "message": {
                "type": "string"
                    }
        }
    }
}
'

Keep in mind that the index name "4glogs" and the type "logs" need to match what is coming from logstash.

For my production systems I generally prefer to turn off dynamic mapping as it avoids accidental mapping creation.

The following links should be useful if you want to make adjustments to your dynamic mappings:

https://www.elastic.co/guide/en/elasticsearch/guide/current/dynamic-mapping.html

http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/custom-dynamic-mapping.html

http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/dynamic-mapping.html