Elasticsearch Nested Filters being inclusive vs. exclusive

Question

I have an object mapping that uses nested objects (props in our example) in a tag-like fashion. Each tag can belong to a client/user and when we want to allow our u

Answer 1

I believe here you might need the advantage of a flattened list of values, like an array of values. The major difference between an array and nested objects is that the latter "knows" which value of a nested property corresponds to another value of another property in the same nested object. The array of values, on the other hand will flatten the values of a certain property and you lose the "association" between a client_id and a name. Meaning, with arrays you have props.client_id = [null, 2] and props.name = ["petlover", "premiumshopper"].

With your nested filter you want to match that string to all values for props.name meaning ALL nested props.names of one parent doc needs to match. Well, this doesn't happen with nested objects, because the nested documents are separate and are queried separately. And, if at least one nested document matches then it's considered a match.

In other words, for a query like "query": "props.name:(carlover NOT petlover)" you basically need to run it against a flattened list of values, just like arrays. You need that query ran against ["carlover", "petlover"].

My suggestion for you is to make your nested documents "include_in_parent": true (meaning, keep in parent a flattened, array-like list of values) and change a bit the queries:

• for the query_string part, use the flattened properties approach to be able to match your query for a combined list of elements, not element by element.
• for the match (or term, see below) and missing parts use the nested properties approach because you can have nulls in there. A missing on an array will match only if the whole array is missing, not one value in it, so here one cannot use the same approach as for the query, where the values were flattened in an array.
• optional, but for the query match integer I would use term, as it's not string but integer and is by default not_analyzed.

These being said, with the above changes, these are the changes:

{
  "mappings" : {
    ...
        "props": {
          "type": "nested",
          "include_in_parent": true,
   ...

1. should (and does) return zero results

GET /nesting-test/_search?pretty=true
{
  "query": {
    "filtered": {
      "filter": {
        "and": [
          {
            "query": {
              "query_string": { "query": "props.name:((carlover AND premiumshopper) NOT petlover)" }
            }
          },
          {
            "nested": {
              "path": "props",
              "filter": {
                "or": [ { "query": { "match": { "props.client_id": 1 } } }, { "missing": { "field": "props.client_id" } } ]
              }
            }
          }
        ]
      }
    }
  }
}

2. should (and does) return just 1

GET /nesting-test/_search?pretty=true
{
  "query": {
    "filtered": {
      "filter": {
        "and": [
          {"query": {"query_string": { "query": "props.name:(carlover NOT petlover)" } } },
          {
            "nested": {
              "path": "props",
              "filter": {
                "or": [{ "query": { "match": { "props.client_id": 1 } } },{ "missing": { "field": "props.client_id" } } ]
              }
            }
          }
        ]
      }
    }
  }
}

3. should (and does) return just 2

GET /nesting-test/_search?pretty=true
{
  "query": {
    "filtered": {
      "filter": {
        "and": [
          { "query": {"query_string": { "query": "props.name:(* NOT carlover)" } } },
          {
            "nested": {
              "path": "props",
              "filter": {
                "or": [{ "query": { "term": { "props.client_id": 1 } } },{ "missing": { "field": "props.client_id" } }
                ]
              }
            }
          }
        ]
      }
    }
  }
}