Substring search in RavenDB

Question

I have a set of objects of type Idea

public class Idea
{
    public string Title { get; set; }
    public string Body { get; set; }
}

Answer 1

Incase anyone else comes across this. Raven 3 has a Search() extension method that allows for substring searching.

A couple of gotchas:

• Pay special attention to the "Query escaping" section at the bottom
• I didn't see it mentioned anywhere, but it only worked for me if Search() was the added directly to Query() (i.e. without any Where(), OrderBy(), etc between them)

Hope this saves someone some frustration.

Answer 2

You can perform substring search across multiple fields using following approach:

( 1 )

public class IdeaByBodyOrTitle : AbstractIndexCreationTask<Idea>
{
    public IdeaByBodyOrTitle()
    {
        Map = ideas => from idea in ideas
                       select new
                           {
                               idea.Title,
                               idea.Body
                           };
    }
}

on this site you can check, that:

"By default, RavenDB uses a custom analyzer called LowerCaseKeywordAnalyzer for all content. (...) The default values for each field are FieldStorage.No in Stores and FieldIndexing.Default in Indexes."

So by default, if you check the index terms inside the raven client, it looks following:

Title                    Body
------------------       -----------------
"the idea title 1"       "the idea body 1"
"the idea title 2"       "the idea body 2" 

Based on that, wildcard query can be constructed:

var wildquery = string.Format("*{0}*", QueryParser.Escape(query));

which is then used with the .In and .Where constructions (using OR operator inside):

var ideas = session.Query<User, UsersByDistinctiveMarks>()
                   .Where(x => x.Title.In(wildquery) || x.Body.In(wildquery));

( 2 )

Alternatively, you can use pure lucene query:

var ideas = session.Advanced.LuceneQuery<Idea, IdeaByBodyOrTitle>()
                   .Where("(Title:" + wildquery + " OR Body:" + wildquery + ")");

( 3 )

You can also use .Search expression, but you have to construct your index differently if you want to search across multiple fields:

public class IdeaByBodyOrTitle : AbstractIndexCreationTask<Idea, IdeaByBodyOrTitle.IdeaSearchResult>
{
    public class IdeaSearchResult
    {
        public string Query;
        public Idea Idea;
    }

    public IdeaByBodyOrTitle()
    {
        Map = ideas => from idea in ideas
                       select new
                           {
                               Query = new object[] { idea.Title, idea.Body },
                               idea
                           };
    }
}

var result = session.Query<IdeaByBodyOrTitle.IdeaSearchResult, IdeaByBodyOrTitle>()
                    .Search(x => x.Query, wildquery, 
                            escapeQueryOptions: EscapeQueryOptions.AllowAllWildcards,
                            options: SearchOptions.And)
                    .As<Idea>();

summary:

Also have in mind that *term* is rather expensive, especially the leading wildcard. In this post you can find more info about it. There is said, that leading wildcard forces lucene to do a full scan on the index and thus can drastically slow down query-performance. Lucene internally stores its indexes (actually the terms of string-fields) sorted alphabetically and "reads" from left to right. That’s the reason why it is fast to do a search for a trailing wildcard and slow for a leading one.

So alternatively x.Title.StartsWith("something") can be used, but this obviously do not search across all substrings. If you need fast search, you can change the Index option for the fields you want to search on to be Analyzed but it again will not search across all substrings.

If there is a spacebar inside of the substring query, please check this question for possible solution. For making suggestions check http://architects.dzone.com/articles/how-do-suggestions-ravendb.

Answer 3

I managed to do this in memory with the following code:

public virtual ActionResult Search(string term)
{
    var clientNames = from customer in DocumentSession.Query<Customer>()
                        select new { label = customer.FullName };

    var results = from name in clientNames.ToArray()
                    where name.label.Contains(term,
                                             StringComparison.CurrentCultureIgnoreCase)
                    select name;

    return Json(results.ToArray(), JsonRequestBehavior.AllowGet);
}

This saved me the trouble of going RavenDB way of searching for strings with Contains method as described by Daniel Lang's post.

The Contains extension method is this:

public static bool Contains(this string source, string toCheck, StringComparison comp)
{
     return source.IndexOf(toCheck, comp) >= 0;
}

Answer 4

This appears to be a duplicate of RavenDB fast substring search

The answer there, which was not mentioned here, is to use a custom Lucene analyzer called NGram