figuring out reason for maxClauseCount is set to 1024 error

Question

I\'ve two sets of search indexes. TestIndex (used in our test environment) and ProdIndex(used in PRODUCTION environment). Lucene search query: +date:[20090410184806 TO 200910071

Answer 1

I had the same problem. My solution was to catch BooleanQuery.TooManyClauses and dynamically increase maxClauseCount.

Here is some code that is similar to what I have in production.

private static Hits searchIndex(Searcher searcher, Query query) throws IOException
{
    boolean retry = true;
    while (retry)
    {
        try
        {
            retry = false;
            Hits myHits = searcher.search(query);
            return myHits;
        }
        catch (BooleanQuery.TooManyClauses e)
        {
            // Double the number of boolean queries allowed.
            // The default is in org.apache.lucene.search.BooleanQuery and is 1024.
            String defaultQueries = Integer.toString(BooleanQuery.getMaxClauseCount());
            int oldQueries = Integer.parseInt(System.getProperty("org.apache.lucene.maxClauseCount", defaultQueries));
            int newQueries = oldQueries * 2;
            log.error("Too many hits for query: " + oldQueries + ".  Increasing to " + newQueries, e);
            System.setProperty("org.apache.lucene.maxClauseCount", Integer.toString(newQueries));
            BooleanQuery.setMaxClauseCount(newQueries);
            retry = true;
        }
    }
}