How to stop/shut down an elasticsearch node?

Question

I want to restart an elasticsearch node with a new configuration. What is the best way to gracefully shut down an node?

Is killing the process the best way of shutting t

Answer 1

Stopping the service and killing the daemon are indeed the correct ways to shutdown a node. However, it's not recommended to do so directly if you want to take down a node for maintenance. In fact, if you don't have replicas you will lose data.

When you directly shutdown a node, Elasticsearch will wait for 1m (default time) for it to come back online. If it doesn't, then it will start to allocate the shards from that node to other nodes wasting lots of IO.

A typical approach would be to disable shard allocation temporarily by issuing:

PUT _cluster/settings
{
  "persistent": {
    "cluster.routing.allocation.enable": "none"
  }
}

Now, when you take down a node, ES won't try to allocate shard from that node to other nodes and you can perform you maintenance activity and then once the node is up, you can enable shard allocation again:

PUT _cluster/settings
{
  "persistent": {
    "cluster.routing.allocation.enable": "all"
  }
}

Source: https://www.elastic.co/guide/en/elasticsearch/reference/5.5/restart-upgrade.html

If you don't have replicas for all your indexes, then performing this type of activity will have downtime on some of the indexes. A cleaner way in this case would be to migrate all the shards to other nodes before taking the node down:

PUT _cluster/settings
{
  "transient" : {
    "cluster.routing.allocation.exclude._ip" : "10.0.0.1"
  }
}

This will move all shards from 10.0.0.1 to other nodes (will take time depending on the data). Once everything is done, you can kill the node, perform maintenance and get it back online. This is a slower operation and is not required if you have replicas.

(Instead of _ip, _id, _name with wildcards will work just fine.)

More information: https://www.elastic.co/guide/en/elasticsearch/reference/5.5/allocation-filtering.html

Other answers have explained how to kill a process.