How to detect language of text?

Question

I have a form which lets users input text snippets. So how can figure out the language of the entered text?

Specifically these languages for now:

Arabic: هذه

Answer 1

Presumably guessing the user's language is to display responses in the proper language. What about examining the browser's settings for preferred languages? Obtain that from the HTTP header Accept-Language. See section 14.4 here.

Answer 2

I'm exploring the same thing, for server-side. Thus far I have found https://code.google.com/p/language-detection/. Hope this helps someone.

Answer 3

You could use the Google Ajax API for detecting the language of a snippet of text.

Answer 4

You can figure out whether the characters are from the Arabic, Chinese, or Japanese sections of the Unicode map.

If you look at the list on Wikipedia, you'll see that each of those languages has many sections of the map. But you're not doing translation, so you don't need to worry about every last glyph.

For example, your Chinese text begins (in hex) 0x8FD9 0x662F 0x4E00 - and those are all in the "CJK Unified Ideographs" section, which is Chinese. Here are a few ranges to get you started:

Arabic (0600–06FF)

Japanese

• Hiragana (3040–309F)
• Katakana (30A0–30FF)
• Kanbun (3190–319F)

Chinese

• CJK Unified Ideographs (4E00–9FFF)

(I got the hex for your Chinese by using a Chinese to Unicode Converter.)

Answer 5

You could use https://detectlanguage.com/ which is a webservice build around CLD2.