surrogate-pairs

Trying to rephrase: Can you map every combining character combination into one code point? I'm new to Unicode, but it seems to me that there is no encoding, normalization or representation where one character would be one code point in every case in Unicode. Is this correct? Is this true for Basic Multilingual Plane also? If you mean one char == one number (ie: where every char is represented by the same number of bytes/words/what-have-you): in UCS-4, each character is represented by a 4-byte number. That's way more than big enough for every character to be represented by a single value, but

Say I have a surrogate pair. For example: \u306f\u30fc Is there a function I can use to print the character to the screen? If you want to do it manually: echo chr(0x30) . chr(0x6f) . chr(0x30) . chr(0xfc); If you have the string, you could always do: $callback = function($match) { return chr(hexdec($match[1])) . chr(hexdec($match[2])); } preg_replace_callback('#\\\\u([0-9a-f]{2})([0-9a-f]{2})#', $callback, $string); Or, if php < 5.3 $callback = create_function('$match', 'return chr(hexdec($match[1])) . chr(hexdec($match[2]));' ); preg_replace_callback('#\\\\u([0-9a-f]{2})([0-9a-f]{2})#',

In Unicode what is the difference between composite characters and surrogate pairs? To me they sound like similar things - two characters to represent one character. What differentiates these two concepts? Kerrek SB Surrogate pairs are a weird wart in Unicode. Unicode itself is nothing other than an abstract assignment of meaning to numbers. That's what an encoding is. Capital-letter-A, Greek-alternate-terminal-sigma, Klingon-closing-bracket-2, etc. currently, numbers up to about 2 21 are available, though not all are in use. In the context of Unicode, each number is know as a code point .

Specifically in Python 3.3 and above, is it sufficient to check for orphan surrogates by using the simple match: re.search(r'[\uD800-\uDFFF]', s) Based on the assumption that all legal surrogates would have been represented as astral code points and thus would not match, leaving out the illegal surrogates, or is there caveats and edge cases one needs to be aware of? Yes, that's correct. Code units 0xD800–0xDFFF don't represent valid characters in wide Unicode strings, and in Python 3.3+ (following PEP 393) all Unicode strings are effectively wide. 来源： https://stackoverflow.com/questions

I have written this method to reverse a string public string Reverse(string s) { if(string.IsNullOrEmpty(s)) return s; TextElementEnumerator enumerator = StringInfo.GetTextElementEnumerator(s); var elements = new List<char>(); while (enumerator.MoveNext()) { var cs = enumerator.GetTextElement().ToCharArray(); if (cs.Length > 1) { elements.AddRange(cs.Reverse()); } else { elements.AddRange(cs); } } elements.Reverse(); return string.Concat(elements); } Now, I don't want to start a discussion about how this code could be made more efficient or how there are one liners that I could use instead. I

Consider the following exchange on IPython: In [1]: s = u'華袞與緼𦅷同歸' In [2]: len(s) Out[2]: 8 The correct output should have been 7 , but because the fifth of these seven Chinese characters has a high Unicode code-point, it is represented in UTF-8 by a "surrogate pair", rather than just one simple codepoint, and as a result Python thinks it is two characters rather than one. Even if I use unicodedata , which returns the surrogate pair correctly as a single codepoint ( \U00026177 ), when passed to len() the wrong length is still returned: In [3]: import unicodedata In [4]: unicodedata.normalize(