What's wrong with this Solr range filter query?

Question

The following filter query returns zero results (using *:* as query):

-startDate:[* TO *] OR startDate:[* TO NOW/DAY+1DAY]

But if I filter only by:

-startDate:[* TO *]

I get 3 results.

If I filter only by:

startDate:[* TO NOW/DAY+1DAY]

I get 161 reults.

Why is the combined FQ returning zero results? What I want is the filter to return any doc whose start date is null or start date is before today.

EDIT:

I'm using Solr 4.2.1.2013.03.26.08.26.55

EDIT:

Well, strange it may sound a colleague suggested putting parenthesis on the two parts like this:

(-startDate:[* TO *]) OR (startDate:[* TO NOW/DAY+1DAY])

And somehow it worked. I'm still curious why that made a difference. Hope someone can shed some light.

Thanks!

Answer 1

Solr supports pure negative queries. They do this, essentially, by expanding the pure negative to something like:

*:* -startDate:[* TO *]

However, what you combine it in a BooleanQuery, I don't believe it applies this sort of logic anymore. A negative query does not, in lucene, fetch anything, but rather filters out matches brought in by other, positive, query terms. This differs from SQL queries, which in a sense start with an implicit *:*, or a full table of results, and allow you to pare it down.

I believe your OR is effectively being ignored, since it doesn't, strictly speaking, make sense in context. Generally, OR is just syntactic sugar, I believe (field:this OR field:that is equivalent to field:this field:that).

So, in effect your query is: startDate:[* TO NOW/DAY+1DAY] -startDate:[* TO *], which makes the results you see more obvious. When you wrap it in parentheses, then each term query is treated separately, and you gain access to solr's support of lonely negative queries.

---

A much better idea is to store a default value, if you need to search for unset/null values. *:* and by extension pure negative queries like this have to scan the entire index, and so perform very poorly. Providing a default value will improve performance, and prevent this sort of confusing situation.

Answer 2

I used femtoRgon's answer and was able to construct a query that included a range and blank values.

The following includes all docs with a StartDate on or after 1/1/2014 and all docs without a StartDate.

(StartDate:[2014-01-01T00:00:00Z TO *]) OR (-StartDate:([* TO *]) AND *:*)

The magic is (-StartDate:([* TO *]) AND *:*). This will select the docs without a StartDate.

Answer 3

Pure negative queries don't work, because they are omitting results from nothing.

Try:

: AND -startDate:[* TO *]

Answer 4

When you query with -startDate:[* TO *] you get documents which do not have any data for the startDate field.

When you query for startDate:[* TO NOW/DAY+1DAY] you get documents which have a value less than or equal to NOW/DAY+1DAY in the startDate field.

You could try -startDate:* OR startDate:[* TO NOW/DAY+1DAY]. The first part says documents that do not have a value and the second part says document having value less than or equal to NOW/DAY+1DAY in the startDate field.