ElasticSearch updates are not immediate, how do you wait for ElasticSearch to finish updating it's index?

Question

I\'m attempting to improve performance on a suite that tests against ElasticSearch.

The tests take a long time because Elasticsearch does not update it\'s indexes i

Answer 1

You can also call elasticsearch.Refresh('blog') if you don't want to wait for the cluster refresh interval

Answer 2

Seems to work for me:

els.indices.refresh(index)
els.cluster.health(wait_for_no_relocating_shards=True,wait_for_active_shards='all')

Answer 3

As of version 5.0.0, elasticsearch has an option:

 ?refresh=wait_for

on the Index, Update, Delete, and Bulk api's. This way, the request won't receive a response until the result is visible in ElasticSearch. (Yay!)

See https://www.elastic.co/guide/en/elasticsearch/reference/master/docs-refresh.html for more information.

edit: It seems that this functionality is already part of the latest Python elasticsearch api: https://elasticsearch-py.readthedocs.io/en/master/api.html#elasticsearch.Elasticsearch.index

Change your elasticsearch.update to:

elasticsearch.update(
     index='blog',
     doc_type='blog'
     id=1,
     refresh='wait_for',
     body={
        ....
    }
)

and you shouldn't need any sleep or polling.

Answer 4

If you use bulk helpers you can do it like this:

from elasticsearch.helpers import bulk    
bulk(client=self.es, actions=data, refresh='wait_for')