Why does anyone use an encoding other than UTF-8? [closed]

Answer 1

Well, some do it because their tools are archaic or flawed. Some do it because they don't see a need to support anything other than ASCII. Some do it because they don't know any better.

Those are the usual excuses for not using Unicode.

As for not using UTF-8 specifically there are different reasons. Some systems, like Windows¹ (and stemming from that, .NET) and Java came to be in a time where Unicode was a strict 16-bit code. Therefore, there was really only one encoding: UCS-2, encoding code points directly as 16-bit words.

Later Unicode was expanded to 21 bits because 65536 code points weren't enough anymore. This caused encodings such as UTF-32 and UTF-16 to appear. For systems previously working with UCS-2 the transition to UTF-16 was the easiest and most sensible choice. Windows did that transition back in Ye Olde Days of Windows 2000.

So while I think that nearly all application nowadays should support Unicode I don't think it is entirely necessary for them to specifically use UTF-8. There are historic reasons for that and no real benefit in converting existing systems from UTF-16 to UTF-8.

---

¹ NT.

Answer 2

Because outside the English-speaking world, people have been using various encodings that predate Unicode and are tailored for their respective languages for decades. These language-specific encodings have become ingrained everywhere and are pretty much a standard. If you want to have any hope of interfacing with legacy systems, you have to use them, so all systems have to support them and usually use them as default even if they by now support UTF-8 as well. There may even be multiple legacy encodings traditionally used for different purposes.

Examples:

• ISO-8859-1 in western Europe - actually outdated there as well, as you need ISO-8859-15 for the Euro sign
• ISO-2022-JP in Japan for emails, Shift JIS for websites
• Big5 in Taiwan
• GB2312 in China

The last two examples show that encodings can even be a political issue.

Answer 3

Many APIs require other Unicode encodings - mostly UTF-16. For instance, Java, .NET, Win32.

Answer 4

Its also worth remembering that in some circumstances (where a non-latin set of characters are needed) UTF-8 can actually bloat larger than the 16 bit Unicode encoding. In those cases ucs-2 or utf-16 would be a better choice.

Answer 5

Unicode certainly is a good place to work from in most cases, but a developer should be familiar with many different types of character encoding. Certainly ASCII might be used if the set of characters is limited.

What if you're a developer and receiving data from a source that doesn't send UTF-8? There could be lots of interface issues if you don't understand your input.

Joel's article on the must-knows for character encoding is good and worth reading.

Answer 6

Related to the subject, when using MySQL, as if it wasn't complex enough, you get the option the choose which kind of UTF-8 collation you want to use. So what would you use?

UTF-8 general ci or UTF-8 unicode ci?

(I tend to use the UTF-8 variant that is used for the database connection)