Elasticsearch: Save aggregations query into CSV file

不问归期 提交于 2020-06-29 03:55:06

问题


I have data such as:

{
  "_index": "user_log",
  "_type": "logs",
  "_id": "gdUJpXIBAoADuwvHTK29",
  "_score": 1,
  "_source": {
    "user_id": 105,
    "user_name": "prathameshsalap@gmail.com",
    "working_hours": "2019-10-21 09:00:01",
    "date": "2019-10-21",
    "working_minutes": 540   
}

{
  "_index": "user_log",
  "_type": "logs",
  "_id": "gtUJpXIBAoADuwvHTK29",
  "_version": 1,
  "_score": 0,
  "_source": {
    "user_id": 106,
    "user_name": "vaishusawant143@gmail.com",
    "working_hours": "2019-10-21 09:15:01",
    "date": "2019-10-21",
    "working_minutes": 555
}

Here I have multiple fields like user_id, user_name, working_hours, date, working_minutes. I only want to choose user_id, user_name, and avg_hours (after calculating avg_working_minutes for each user).

body = {
    "query" : {"match_all": {}},
     "aggs": {
      "users": {
          "terms": {
              "field": "user_name.keyword",
              "order": {
                 "avg_hours": "desc"
              }
          },
          "aggs": {
              "avg_hours": {
                  "avg": {
                      "field": "working_minutes"
                  }
              }
          }
      }
  }
}

es_obj = Elasticsearch()
response = els_obj.search(index='user_log', body)
# save into csv
with open(path, 'w') as files:
    header_present = False
    for doc in response['hits']['hits']:
        my_dict = doc['_source'] 
        if not header_present:
           w = csv.DictWriter(files, my_dict.keys())
           w.writeheader()
           header_present = True
        w.writerow(my_dict)

It's return output like:

user_id  |  date    |         user_name       |  working_hours,   | working_minutes
---------|----------|-------------------------|-------------------|----------
  105    |2019-10-21|prathameshsalap@gmail.com|2019-10-21 09:00:01|   540
  106    |2019-10-21|vaishusawant143@gmail.com|2019-10-21 09:15:01|   555

Here It returns all user_id, user_name, working_hours, date, working_minutes. I don't want all of those fields. So, how to select multiple fields(user_id, user_name, and avg_hours) in this query?

expected output:

user_id  |         user_name       | Avg_hour
---------|-------------------------|----------
  105    |prathameshsalap@gmail.com|   450.55
  106    |vaishusawant143@gmail.com|   350

来源:https://stackoverflow.com/questions/62349069/elasticsearch-save-aggregations-query-into-csv-file

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!