Dump all documents of Elasticsearch

Question

Is there any way to create a dump file that contains all the data of an index among with its settings and mappings?

A Similar way as mongoDB does with mongodump

Answer 1

For your case Elasticdump is the perfect answer.
First, you need to download the mapping and then the index

# Install the elasticdump 
npm install elasticdump -g

# Dump the mapping 
elasticdump --input=http://<your_es_server_ip>:9200/index --output=es_mapping.json --type=mapping

# Dump the data
elasticdump --input=http://<your_es_server_ip>:9200/index --output=es_index.json --type=data    

If you want to dump the data on any server I advise you to install esdump through docker. You can get more info from this website Blog Link

Answer 2

ElasticSearch itself provides a way to create data backup and restoration. The simple command to do it is:

CURL -XPUT 'localhost:9200/_snapshot/<backup_folder name>/<backupname>' -d '{
    "indices": "<index_name>",
    "ignore_unavailable": true,
    "include_global_state": false
}'

Now, how to create, this folder, how to include this folder path in ElasticSearch configuration, so that it will be available for ElasticSearch, restoration method, is well explained here. To see its practical demo surf here.

Answer 3

Here's a new tool we've been working on for exactly this purpose https://github.com/taskrabbit/elasticsearch-dump. You can export indices into/out of JSON files, or from one cluster to another.

Answer 4

Elasticsearch supports this now out of the box:

https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-snapshots.html

Answer 5

You can also dump elasticsearch data in JSON format by http request: https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-scroll.html
CURL -XPOST 'https://ES/INDEX/_search?scroll=10m'
CURL -XPOST 'https://ES/_search/scroll' -d '{"scroll": "10m", "scroll_id": "ID"}'

Answer 6

To export all documents from ElasticSearch into JSON, you can use the esbackupexporter tool. It works with index snapshots. It takes the container with snapshots (S3, Azure blob or file directory) as the input and outputs one or several zipped JSON files per index per day. It is quite handy when exporting your historical snapshots. To export your hot index data, you may need to make the snapshot first (see the answers above).