Recognizing language of a short text? [closed]

Answer 1

In general you're looking at doing nGram identification. Since this is a python question, you might take a look at http://github.com/koblas/ngramj-python which is a pure python port of the java ngram library (another open source project).

The documentation is lacking, but it has really good accuracy.

Answer 2

I know this is an old question, but in case people come across this while researching options for this task, it is worth mentioning that another tool is langid.

Answer 3

I'd use the guess-language project.

Edit: Now in Bitbucket

Answer 4

You could try the Google AJAX Language API if you don't mind using a web service to do your work for you.

Answer 5

Have you looked into http://ling.unizd.hr/~dcavar/LID/ and http://en.wikipedia.org/wiki/Language_identification ?

Answer 6

If neos recommendation is also unpractical, I would try something like this:

In many languages there are some keywords which are in many sentences and are often not found in other languages.

Example: "The" in English, "der", "die", "das" in German, ....

Find such words and try to find them in your texts. It can be a little fuzzy at last -- for example, when you find "the" and "der" -- it could be a German text containing some English sentences in it. At least with enough words from your target languages you could come to a high hit-rate.