ElasticSearch

问题 Using the Elasticsearch scan-and-scroll feature, is it possible to control both the size of the batches returned, as well as the limit on the number of matches? According to the Elasticsearch scan-and-scroll documentation: Although we specified a size of 1,000, we get back many more documents. When scanning, the size is applied to each shard, so you will get back a maximum of size * number_of_primary_shards documents in each batch. This seems to indicate that the size parameter is used

问题 Using the Elasticsearch scan-and-scroll feature, is it possible to control both the size of the batches returned, as well as the limit on the number of matches? According to the Elasticsearch scan-and-scroll documentation: Although we specified a size of 1,000, we get back many more documents. When scanning, the size is applied to each shard, so you will get back a maximum of size * number_of_primary_shards documents in each batch. This seems to indicate that the size parameter is used

问题 Using the Elasticsearch scan-and-scroll feature, is it possible to control both the size of the batches returned, as well as the limit on the number of matches? According to the Elasticsearch scan-and-scroll documentation: Although we specified a size of 1,000, we get back many more documents. When scanning, the size is applied to each shard, so you will get back a maximum of size * number_of_primary_shards documents in each batch. This seems to indicate that the size parameter is used

问题 I have 7 nodes in my es cluster with the same cpu and memory. 5 of them have three 1T disks, the other 2 have only one 1T disk. I find that the shards on my cluster frequently relocate, but the disk usage is very unbalanced, for example as below: usage: 34% free: 1.94TB total: 2.95TB usage: 55% free: 1.33TB total: 2.95TB usage: 57% free: 1.27TB total: 2.93TB usage: 46% free: 1.60TB total: 2.95TB usage: 55% free: 1.31TB total: 2.93TB usage: 90% free: 100.40GB total: 1007.93GB usage: 85% free:

问题 I have just set up a new ElasticSearch cluster on ElasticCloud and I am trying to follow the help documentation. It says you can post a document as follows: curl https://<id>.<region>.aws.found.io:9243/my_index/my_type -XPOST -d '{"title": "One", "tags": ["ruby"] }' So I am trying this with my own URL, but I am getting: {"error":{"root_cause":[{"type":"security_exception","reason":"action [indices:data/write/index] requires authentication","header":{"WWW-Authenticate":"Basic realm=\"security\

问题 I have just set up a new ElasticSearch cluster on ElasticCloud and I am trying to follow the help documentation. It says you can post a document as follows: curl https://<id>.<region>.aws.found.io:9243/my_index/my_type -XPOST -d '{"title": "One", "tags": ["ruby"] }' So I am trying this with my own URL, but I am getting: {"error":{"root_cause":[{"type":"security_exception","reason":"action [indices:data/write/index] requires authentication","header":{"WWW-Authenticate":"Basic realm=\"security\

问题 Getting FATAL Error when starting Kibana 7.5, i have not configured anything i am just running elasticsearch.bat (also v7.5) and kibana.bat '{"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"No handler for type [flattened] declared on field [state]"}],"type":"mapper_parsing_exception", "reason":"Failed to parse mapping [_doc]: No handler for type [flattened] declared on field [state]", "caused_by":{"type":"mapper_parsing_exception","reason":"No handler for type [flattened]

问题 Getting FATAL Error when starting Kibana 7.5, i have not configured anything i am just running elasticsearch.bat (also v7.5) and kibana.bat '{"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"No handler for type [flattened] declared on field [state]"}],"type":"mapper_parsing_exception", "reason":"Failed to parse mapping [_doc]: No handler for type [flattened] declared on field [state]", "caused_by":{"type":"mapper_parsing_exception","reason":"No handler for type [flattened]

问题 I want to query IP range from:172.16.0.0 to 172.31.0.0 in ELK I try two query methods, but fail. { "query": { "bool": { "should": [ { "regexp": { "DstIP": "172.(3[0-1]|1[6-9]|2[0-9]).*" } } ], "minimum_should_match": 1 } } } { "query": { "range": { "DstIP": { "gte": "172.16.0.0", "lte": "172.31.0.0" } } } } How can query IP range in ELK? 回答1: For range queries to work correctly on IP values it is necessary to define the field data type as ip. Below is the working example with mapping, sample

问题 I am going to work on some projects to deal with entity deduplication. Datasets (one or more) which may contain duplicate entity. In the realtime, entity may represent the name, address, country, email, social media id in the different form. My goal is to identify that these are possible duplicates based on different weightage for the different entity Info. I am trying to look for a library that is open-source & preferably written in Java. As I need to process the millions of data, I need to