Preferred method of indexing bulk data into ElasticSearch?

Question

I\'ve been looking at ElasticSearch as solution get some better search and analytics functionality at my company. All of our data is in SQL Server at the moment and I\'ve succes

Answer 1

There are a lot of things that you can do. You can put your data in rabbitmq or redis, but your main problem is staying up to date. I guess you should look into an event based application. But if you really only have the sql server as a datasource you could work with timestamps and a query that checks for updates. Depending on the size of your database you can also just reindex the complete dataset.

Using events or the query based solution, you can push these updates to elasticsearch, probably using the bulk api.

The good part about a custom solution like this is that you can think about your mapping. This is important if you really want to do something smart with your data.