Delete documents of type in Elasticsearch

Question

I want to delete all the documents indexed within a type in Elasticsearch, using the HTTP/REST api, but I don\'t want to delete the mapping for this type

How can I build

Answer 1

If you want to do this in golang, using the "olviere/elastic" library, you can use this code, assuming you have a client yourClient, and yourIndex and yourType:

    bq := elastic.NewBoolQuery()
    bq.Must(elastic.NewMatchAllQuery())
    _, err := elastic.NewDeleteByQueryService(yourClient).
        Index(yourIndex).
        Type(yourType).
        Query(bq).
        Do()

Answer 2

Previous answers will not work with the most recent version of Elasticsearch. "Delete by query" was deprecated from Elasticsearch 2.0. Elasticsearch documentation says that it can cause an OutOfMemoryError during concurrent indexing and can cause primary and replica to become inconsistent. If you want follow the history of the issue in Github.

It now takes multiple steps in order to delete all documents from a type.

1. Find all the ids of the document that you need to delete. The most efficient way to perform this operation is to use the scroll/scan API to find all the matching ids for a given type.

2. Issue a bulk request to delete the documents by ids. An example provided below.

   curl -XPOST 'http://localhost:9200/_bulk' -d '
       { "delete": { "_index": "index", "_type": "type", "_id": "1"}
       { "delete": { "_index": "index", "_type": "type", "_id": "2"}'
   

Note that if you are providing a text file input to curl, you must use the --data-binary flag instead of plain -d.

Answer 3

$ curl -XDELETE 'http://localhost:9200/twitter/tweet/_query?q=user:kimchy'

$ curl -XDELETE 'http://localhost:9200/twitter/tweet/_query' -d '{
    "query" : {
        "term" : { "user" : "kimchy" }
    }
}

Answer 4

A simple delete by query with a match_all query should do the trick. You can grab more info here :

delete by query api

Alternatively, you can delete the whole type and make use of the template api. Just drop a file in your config/templates/ folder containing your template, and you'll never loose it. The mapping will indeed be lost when you'll delete the mapping, but the template will be reused as soon as you index something again. Here's some more info :

template api

EDIT: new delete api: https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-delete.html

Answer 5

With the following command in the elasticsearch head plugin I was able to delete all documents of type logs from the logstash index without deleting the mapping:

{"query":{"match_all":{}}}

To free space on the disk you must also optimize the index (Actions->Optimize for index logstash in the head plugin) after deleting the documents.

Answer 6

Before executing command, index/mapping state; (screenshots taken from elasticsearch head plugin web interface)

Command;

curl -XDELETE 'http://localhost:9200/publishercategoryeu/autocomplete/_query' -d '
{
  "query": {
    "bool": {
      "must": [
        {
          "match_all": {}
        }
      ]
    }
  }
}
'

Result;

After executing command, index/mapping state;

As we can see we deleted all the documents indexed within a type(mapping) without delete index or type(mapping).