Sphinx and “did you mean … ?” suggestions idea. WIll it work?

Question

I\'m trying to come up with the fastest way to make search suggestions. At first I thought a Levenstein UDF function combined with a mysql table would do the job. But using

Answer 1

I think it will be interesting to you to read what Andrew Aksyonoff (author of Sphinx) thinks about implementation of this task via Sphinx - http://habrahabr.ru/blogs/sphinx/61807/ (use translator to translate from russian)

Answer 2

I can't see a problem with your idea. Go for it. Just to point out that your method is only relevant if you want to override the builtin behaviour that is very similar to LD.

For example, with sphinx 1.10-beta, you can specify min_infix_len and expand_keywords and use sphinx's builtin weighting methods (BM25 and some proprietary code) for good results. http://sphinxsearch.com/blog/2010/08/17/how-sphinx-relevance-ranking-works/

Don't forget to memcache these queries, and create a warm-up script.

Answer 3

You could just log every search query that's entered, along with a next search query that the user enters.

Lets assume that lots of users search for rhinosorous but actually mean rhinoceros. Because users will correct their query, this will mean there will be a lot of rhinosorous queries with rhinoceros as the next query.

You can select suggestions like this:

SELECT id, query, next_query, COUNT(id) AS count FROM queries GROUP BY query ORDER BY COUNT(id) DESC

If the top result has a count that's a high % of all queries for that keyword, display a message.

I haven't tested this, its just an idea.