Python converting latin1 to UTF8

核能气质少年 提交于 2019-12-04 02:34:06

To decode a byte sequence from latin 1 to Unicode, use the .decode() method:

>>> '\xe9'.decode('latin1')
u'\xe9'

Python uses \xab escapes for unicode codepoints below \u00ff.

>>> '\xe9'.decode('latin1') == u'\u00e9'
True

The above Latin-1 character can be encoded to UTF-8 as:

>>> '\xe9'.decode('latin1').encode('utf8')
'\xc3\xa9'
>>> u"é".encode('utf-8')
'\xc3\xa9'

You've got a UTF-8 encoded byte sequence. Don't try to print encoded bytes directly. To print them you need to decode the encoded bytes back into a Unicode string.

>>> u"é".encode('utf-8').decode('utf-8')
u'\xe9'
>>> print u"é".encode('utf-8').decode('utf-8')
é

Notice that encoding and decoding are opposite operations which effectively cancel out. You end up with the original u"é" string back, although Python prints it as the equivalent u'\xe9'.

>>> u"é" == u'\xe9'
True

concept = concept.encode('ascii', 'ignore') concept = MySQLdb.escape_string(concept.decode('latin1').encode('utf8').rstrip())

I do this, I am not sure if that is a good approach but it works everytime !!

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!