Emoji characters cannot be encoded to JSON

Question

I have a UITextView which I call messageField. The data within that messageField is POST-ed to server in JSON format. When th

Answer 1

One thing to be aware of with Emoji - if your json encoder is set to produce ASCII output (i.e. using the \u<4 hex digits> character format), some Emoji characters will break because they use more than two bytes and thus don't fit in the 4 hex digits allowed.

So in python for example, be sure to use:

json.dumps(<object containing emoji strings>, ensure_ascii=False)

Answer 2

Update Swift & Objective C

If you want to send send an emoji item through JSON, first you need to formate DB to UTF-8 AND in IOS you need to encode for NSUTF8StringEncoding. So first make sure your DB formate to UTF-8 then encode parameters to NSUTF8StringEncoding.So here is a sample request when sending the message.

    let post:NSString = parameters!
    let url:NSURL = NSURL(string: serverURL!)!
    let postData:NSData = post.dataUsingEncoding(NSUTF8StringEncoding)! // Change Here
    let postLength:NSString = String( postData.length )
    let request:NSMutableURLRequest = NSMutableURLRequest(URL: url)
    request.HTTPMethod = "POST"
    request.HTTPBody = postData
    request.setValue(postLength as String, forHTTPHeaderField: "Content-Length")
    request.setValue("application/x-www-form-urlencoded", forHTTPHeaderField: "Content-Type")
    request.setValue("*/*", forHTTPHeaderField: "Accept")
    return request

But you don't need to decode to NSUTF8StringEncoding in iOS.Because according to the apple documentation it decode 5 encodings.

The data must be in one of the 5 supported encodings listed in the JSON specification: UTF-8, UTF-16LE, UTF-16BE, UTF-32LE, UTF-32BE. The data may or may not have a BOM. The most efficient encoding to use for parsing is UTF-8, so if you have a choice in encoding the data passed to this method, use UTF-8.

https://developer.apple.com/library/ios/documentation/Foundation/Reference/NSJSONSerialization_Class/#//apple_ref/occ/clm/NSJSONSerialization/JSONObjectWithData:options:error:

Answer 3

I use the below code to encode emoji character

NSString *uniText = [NSString stringWithUTF8String:[textview.text UTF8String]]; 
NSData *msgData = [uniText dataUsingEncoding:NSNonLossyASCIIStringEncoding]; 
NSString *goodMsg = [[NSString alloc] initWithData:msgData encoding:NSUTF8StringEncoding] ;

And the below code to decode and display in UILabel

const char *jsonString = [body UTF8String]; 
NSData *jsonData = [NSData dataWithBytes:jsonString length:strlen(jsonString)]; 
NSString *goodMsg = [[NSString alloc] initWithData:jsonData encoding:NSNonLossyASCIIStringEncoding];

Answer 4

I had the same problem, after digging for hours and finally found this answer that works for me: https://stackoverflow.com/a/8339255/1090945

If you are using rails as your server, this is all you need to do. No need to do anything in ios/xcode, just pass the NSString without doing any UTF8/16 encoding stuff to the server.

Postegre stores the code correctly, it's just when you send the json response back to your ios client, assuming you do render json:@message, the json encoding has problem.

you could test whether you are having json encoding problem in your rails console by doing as simple test:

test = {"smiley"=>"u{1f604}"} test.to_json

if it prints out "{\"smiley\":\"\uf604\"}" (notice the 1 is lost), then you have this problem. and the patch from the link will fix it.

Answer 5

Edit - 2016-03-03 Please note, this answer was written in 2011 and may no longer be relevant any more.

Emoji characters are just a specific font used to render specific unicode code points. iOS uses one of the Unicode Private Use Areas for Emoji-specific code points. The only way to view these "characters" as Emoji is to have an Emoji font available as well as a machine that knows how to switch from the default text font (such as Helvetica) to the emoji font.

I don't know how you're encoding your JSON but since Emoji are just text there shouldn't be any problems as long as you transport the text in a Unicode-friendly format such as UTF-8 or UTF-16. You won't see it on the server-side or in the database (unless you view the data with the previous prerequisites) but you should be able to send the same raw bytes back and it should look the same.

Here's some posts that might help a little more:

• The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!)
• Questions about iPhone emoji and web pages
• iPhone Emoji

Answer 6

Thanks @Karu, editing your response,I use this code:

Encode for send to server:

NSString *uniText = [NSString stringWithUTF8String:[text_to_send UTF8String]];

NSData *msgData = [uniText dataUsingEncoding:NSNonLossyASCIIStringEncoding];

NSString *str = [[NSString alloc] initWithData:msgData encoding:NSUTF8StringEncoding];

Decode to receive and showing emoji text:

NSData *newdata = [received_string dataUsingEncoding:NSUTF8StringEncoding allowLossyConversion:YES];

NSString *mystring = [[NSString alloc] initWithData:newdata encoding:NSNonLossyASCIIStringEncoding];

NSString *finalString= @"";

if (mystring) {
   finalString = mystring;
}else{
   finalString = received_string;
}

This code run perfect on iOS 9