Using Solr for indexing multiple languages

Question

We\'re setting up a Solr to index documents where title field can be in various languages. After googling I found two options:

1. Define different schema fields for e

Answer 1

There's also a third alternative where you use a common set of fields for all languages but apply a filter to a field language. For instance if you have the fields text, language you can put text contents for all languages in to the text field and use e.g., fq=language:english to only retrieve english documents.

The downside of this approach is that you cannot use language specific features such as lemmatisation, stemming, etc.

Define different schema fields for every language i.e. title_en, title_fr,... applying different filters to each language then query one of title fields with a corresponding language.

This approach gives good flexibility, but beware of high memory consumption and complexity when many languages are present. This can be mitigated using multiple solr servers.

Creating different Solr cores to handle each language and make our app query correct Solr core.

Definately a nice solution. But whether the separate administration and slight overhead will work for you is probably in relation to the number of languages you wish to use.

Unless the first approach is applicable, I would probably lean towards the second one unless the scalability of cores isn't desired. Either approach is fine though and I think it basicaly comes down to preference.