AWS elastic-search. FORBIDDEN/8/index write (api). Unable to write to index

Question

I am trying dump a list of docs to an AWS elastic-search instance. It was running fine. Then, all of sudden it started throwing this error:

{ _index: \'

        

Answer 1

ES could apply write block on index during rollovers, or Low disk space or memory.

In order to stop these errors you need to remove the write block on the index by setting index.blocks.write to false

curl -X PUT -H "Content-Type: application/json" 'http://localhost:9200/{index_name}/_settings' -d '{ "index": { "blocks": { "write": "false" } } } '

Answer 2

This error is the Amazon ES service actively blocking writes to protect the cluster from reaching red or yellow status. It does this using index.blocks.write.

The two reasons being:

Low Memory

When the JVMMemoryPressure metric exceeds 92% for 30 minutes, Amazon ES triggers a protection mechanism and blocks all write operations to prevent the cluster from reaching red status. When the protection is on, write operations fail with a ClusterBlockException error, new indexes can't be created, and the IndexCreateBlockException error is thrown.

When the JVMMemoryPressure metric returns to 88% or lower for five minutes, the protection is disabled, and write operations to the cluster are unblocked.

Low Disk Space

Elasticsearch has a default "low watermark" of 85%, meaning that once disk usage exceeds 85%, Elasticsearch no longer allocates shards to that node. Elasticsearch also has a default "high watermark" of 90%, at which point it attempts to relocate shards to other nodes.

Answer 3

This error indicates that AWS ElasticSearch has placed a block on your domain based upon disk space. At 85%, ES will not allow you create any new indexes. At 90%, no new documents can be written.

Answer 4

This can also happen if the index you're trying to write to has been marked as read only. I've had it happen due to an Index State Management misconfiguration which caused a weekly index to be moved to a warm state after one day.