How to do “where not exists” type filtering in Kibana/ELK?

Question

I am using ELK to create dashboards from my log files. I have a log file with entries that contain an id value and a \"success\"/\"failure\" value, displaying whether an operati

Answer 1

! (_exists_:NAME) is not working for me. I use suggestion from:

https://discuss.elastic.co/t/kibana-5-0-0--missing--is-not-working-anymore/64336

NOT _exists_:NAME

UPDATE The problem I faced is that ES syntax forbids spaces after negation operators. Use one of:

NOT _exists_:FIELD
!_exists_:FIELD
-_exists_:FIELD

Check tutorial: https://www.timroes.de/2016/05/29/elasticsearch-kibana-queries-in-depth-tutorial/

---

NOTE: In Elasticsearch 7.x, Kibana now has a pull down to select KQL or Lucene style queries in the search bar. Be mindful that syntax such as _exists_:FIELD is a Lucene syntax and you need to set the pulldown accordingly.

Answer 2

This is easy in Kibana 5 search bar. Just add a filter

!(_exists_:"your_variable") 

you can toggle the filter or write the inverse query as

_exists_:"your_variable"

---

In Kibana 4 and Kibana 3 you can use this query which is now deprecated

_missing_:"your_variable"  

---

NOTE: In Elasticsearch 7.x, Kibana now has a pull down to select KQL or Lucene style queries in the search bar. Be mindful that syntax such as _exists_:FIELD is a Lucene syntax and you need to set the pulldown accordingly.

Answer 3

In newer ELK versions (I think after Elasticsearch 6) you should use field:* to check if the field exist and not field:* to check if it's missing.

elastic search reference: https://www.elastic.co/guide/en/elasticsearch/reference/6.5/query-dsl-query-string-query.html#_wildcards

Answer 4

One option would be to create an own query for this criteria in Kibana. Then just have your panel that does the counting just to use this query.

value:failure

More information here: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-query-string-query.html#query-string-syntax