ElasticSearch

Elastic Pipelines: Skip Import On Failure

两盒软妹~` 提交于 2021-02-10 14:40:38
问题 Individual processors in an Elastic pipelines have an on_failure attribute. This allows you to handle a failure/error in a pipeline. The example in the docs show setting an additional field on your document. { "description" : "my first pipeline with handled exceptions", "processors" : [ { "rename" : { "field" : "foo", "to" : "bar", "on_failure" : [ { "set" : { "field" : "error.message", "value" : "{{ _ingest.on_failure_message }}" } } ] } } ] } Is it possible to tell the pipeline to SKIP

Filtering collapsed results in Elasticsearch

Deadly 提交于 2021-02-10 14:36:32
问题 I have an elasticsearch index containing documents that represent entities at a given point in time. When an entity changes state, a new document is created with a timestamp. When I need to get the current state of all entities, I can do the following: GET https://127.0.0.1:9200/myindex/_search { "collapse": { "field": "entity_id" }, "sort" : [{ "timestamp": { "order": "desc" } }] } However, I would like to further filter the result of the collapse. When entities are deleted I create a new

What is the use of maintaining two aliases for a single Elastic Search Index

亡梦爱人 提交于 2021-02-10 14:18:10
问题 I have been exploring Elastic Search lately. I have been going through aliases. I see ES provides an API to create multiple aliases to a single index like below: { "actions" : [{ "add" : { "indices" : ["test1", "test2"], "alias" : "alias1" } }] } Refer: https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-aliases.html#indices-aliases I'm wondering what is the use case of this. Won't the queries on aliases get split if an alias point to multiple indices? I have tried getting

Using Nested Fields in Composite Aggregation

淺唱寂寞╮ 提交于 2021-02-10 14:14:45
问题 Basically, I'm trying to find the duplicate contacts by first name, last name & email address. For that, I've tried to use composite aggregation with the fields firstName, lastName & emails.email, the response from the query has the values bucketed for non-nested fields(such as firstName & lastName), but the nested field emails.email doesn't have value at all -> it returns NULL: https://www.screencast.com/t/98CKr0I5 Am I missing something here? any help would be greatly appreciated. Below is

ElasticSearch composite ID

纵饮孤独 提交于 2021-02-10 13:53:45
问题 I am using elasticsearch-py to insert data. Currently, my _id field is something like this: _id: "{'hostname': 'xxxxx', 'day': '2016-03-22', 'service': 'xxxxxxxx'}" My manager told me this is not a good id. I think it's ok. With ElasticSearch, can i have an id composed of several attributes ? Eg: _id: { hostname: "xxxxxx", day: "2016-03-22", service: "xxxxxxxx" } A second solution is to encode my string in base64 with python. 来源: https://stackoverflow.com/questions/36174474/elasticsearch

Something inside Elasticsearch 7.4 cluster is getting slower and slower with read timeouts now and then

主宰稳场 提交于 2021-02-10 12:46:11
问题 Regularly the past days our ES 7.4 cluster (4 nodes) is giving read timeouts and is getting slower and slower when it comes to running certain management commands. Before that it has been running for more than a year without any trouble. For instance /_cat/nodes was taking 2 minutes yesterday to execute, today it is already taking 4 minutes. Server loads are low, memory usage seems fine, not sure where to look further. Using the opster.com online tool I managed to get some hint that the

Elasticsearch Query on indexes whose name is matching a certain pattern

雨燕双飞 提交于 2021-02-10 06:14:14
问题 I have a couple of indexes in my Elasticsearch DB as follows Index_2019_01 Index_2019_02 Index_2019_03 Index_2019_04 . . Index_2019_12 Suppose I want to search only on the first 3 Indexes. I mean a regular expression like this: select count(*) from Index_2019_0[1-3] where LanguageId="English" What is the correct way to do that in Elasticsearch? 回答1: How can I query several indexes with certain names? This can be achieved via multi-index search, which is a built-in capability of Elasticsearch.

Elasticsearch Query on indexes whose name is matching a certain pattern

社会主义新天地 提交于 2021-02-10 06:14:08
问题 I have a couple of indexes in my Elasticsearch DB as follows Index_2019_01 Index_2019_02 Index_2019_03 Index_2019_04 . . Index_2019_12 Suppose I want to search only on the first 3 Indexes. I mean a regular expression like this: select count(*) from Index_2019_0[1-3] where LanguageId="English" What is the correct way to do that in Elasticsearch? 回答1: How can I query several indexes with certain names? This can be achieved via multi-index search, which is a built-in capability of Elasticsearch.

Elasticsearch using shingle filter with synonym

佐手、 提交于 2021-02-10 05:55:12
问题 I have the following documents: south africa north africa I want to retrieve my "south africa" document from: s africa (a) southafrica (b) safrica (c) I defined the followings filters and analyzers: POST test_index { "settings": { "analysis": { "filter": { "synonym_filter": { "type": "synonym", "synonyms": [ "south,s", "north,n" ] }, "shingle_filter": { "type": "shingle", "min_shingle_size": 2, "max_shingle_size": 3, "token_separator": "" } }, "analyzer": { "my_shingle": { "type": "custom",

Try to query and aggregate in ElasticSearch but aggregrating not working - elasticsearch.js client

走远了吗. 提交于 2021-02-10 05:13:47
问题 I'm trying to query my dataset for two purposes: Match a term (resellable = true) Order the results by their price lowest to highest Data set/doc is: "data" : { "resellable" : true, "startingPrice" : 0, "id" : "4emEe_r_x5DRCc5", "buyNowPrice" : 0.006493, //Changes per object "sub_title" : "test 1", "title" : "test 1", "category" : "Education", } //THREE OBJECTS WITH THE VALUES OF 0.006, 0.7, 1.05 FOR BUYNOWPRICE I have three objects of these with different buyNowPrice Query with agg is: {