Recognising timestamps in Kibana and ElasticSearch

后端 未结 3 1349
囚心锁ツ
囚心锁ツ 2021-02-10 04:05

I\'m new to ElasticSearch and Kibana and am having trouble getting Kibana to recognise my timestamps.

I have a JSON file with lots of data that I wish to insert into Ela

相关标签:
3条回答
  • 2021-02-10 04:32

    There is no need to make and ISO8601 date in case you have an epoch timestamp. To make Kibana recognize the field as date is has to be a date field though.

    Please note that you have to set the field as date type BEFORE you input any data into the /index/type. Otherwise it will be stored as long and unchangeable.

    Simple example that can be pasted into the marvel/sense plugin:

    # Make sure the index isn't there
    DELETE /logger
    
    # Create the index
    PUT /logger
    
    # Add the mapping of properties to the document type `mem`
    PUT /logger/_mapping/mem
    {
      "mem": {
        "properties": {
          "timestamp": {
            "type": "date"
          },
          "free": {
             "type": "long"
          }
        }
      }
    }
    
    # Inspect the newly created mapping
    GET /logger/_mapping/mem
    

    Run each of these commands in serie.

    Generate free mem logs

    Here is a simple script that echo to your terminal and logs to your local elasticsearch:

    while (( 1==1 )); do memfree=`free -b|tail -n 1|tr -s ' ' ' '|cut -d ' ' -f4`; echo $load; curl -XPOST "localhost:9200/logger/mem" -d "{ \"timestamp\": `date +%s%3N`, \"free\": $memfree }"; sleep 1; done
    

    Inspect data in elastic search

    Paste this in your marvel/sense

    GET /logger/mem/_search
    

    Now you can move to Kibana and do some graphs. Kibana will autodetect your date field.

    0 讨论(0)
  • 2021-02-10 04:33

    Managed to solve the problem. So for anyone else having this problem:

    The format we had our date saved in was incorrect, needed to be :

    "_timestamp":"2013-07-05 08:49:30.123"
    

    then our mapping needed to be:

    curl -XPUT 'http://localhost:9200/test/container/_mapping' -d'
    {
    "container" : {
    "_timestamp" : {"enabled": true, "type":"date", "format": "yyyy-MM-dd HH:mm:ss.SSS", "store":true, "path" : "_timestamp"}
    }
    }'
    

    Hope this helps someone.

    0 讨论(0)
  • 2021-02-10 04:49

    This solution works for older ES <2.4 For the newer version of ES you can either use the "date" field along with the following parameters: https://www.elastic.co/guide/en/elasticsearch/reference/current/date.html

    0 讨论(0)
提交回复
热议问题