Are 6 octet UTF-8 sequences valid?

Question

Can UTF-8 encode 5 or 6 byte sequences, allowing all Unicode characters to be encoded? I\'m getting conflicting standards. I need to be able to support every Unico

Answer 1

They are no Unicode characters beyond 10FFFF, the BMP covers 0000 through FFFF.

UTF-8 is well-defined for 0-10FFFF.

Answer 2

Both UTF-8 and UTF-16 allow all Unicode characters to be encoded. What UTF-8 is not allowed to do is to encode upper and lower surrogate halves (which UTF-16 uses) or values above U+10FFFF, which aren't legal Unicode.

Note that the BMP ends at U+FFFF.

Answer 3

I would have to say no: Unicode code points are valid for the range [0, 0x10FFFF], and those map to 1-4 octets. So, if you did come across a 5- or 6-octet UTF-8 encoded code point, it's not a valid code point - there's certainly nothing assigned there. I am a little baffled as to why they're there in the ISO standard - I couldn't find an explanation.

It does make you wonder, however, if perhaps someday in the future, they would expand past U+10FFFF. 0x10FFFF allows for over a million characters, but there are a lot characters out there, and it would depend how much eventually gets encoded. (For sanity's sake, let's hope not, a million characters is a lot!) UTF-32 could handle more code points, and as you've discovered, UTF-8 could. It'd really be UTF-16 that's out of luck - more surrogate pairs would be needed somewhere in the spectrum of code points.