问题
I have a foreign C library which uses utf-16 in API: as function arguments, return values and structure members.
On Windows its OK with ctypes.c_wchar_p, but under OSX ctypes uses UCS-32 in c_wchar and I could not find the way to support utf-16.
Here is my research:
Use _SimpleCData subclassing to redefine _check_retval_.
- it allows a transparent conversion of utf-16 to Python string.
- can be placed as C structure member
- But it doesn't allow to handle strings as arguments, its
from_param()
method never been called (Why?):func('str', b'W\x00B\x00\x00\x00') # passed without conversion
Use own type with
from_param()
method.- advantages: can be initialized either with constructor or encoded on the fly when passing strings to function:
- disadvantages: cannot be used as function return type or structure member.
Here it is:
ustr = myutf16('hello')
func(ustr)
func('hello') # calls myutf16.from_param('hello')
回答1:
You can override from_param
in a c_char_p
subclass to encode a unicode
string as UTF-16. You can add a _check_retval_
method to decode a UTF-16 result as a unicode
string. For struct fields you can use a descriptor class that handles setting and getting the attribute. Make the field a private _name
of type c_char_p
, and set the descriptor as the public name
. For example:
import sys
import ctypes
if sys.version_info[0] > 2:
unicode = str
def decode_utf16_from_address(address, byteorder='little',
c_char=ctypes.c_char):
if not address:
return None
if byteorder not in ('little', 'big'):
raise ValueError("byteorder must be either 'little' or 'big'")
chars = []
while True:
c1 = c_char.from_address(address).value
c2 = c_char.from_address(address + 1).value
if c1 == b'\x00' and c2 == b'\x00':
break
chars += [c1, c2]
address += 2
if byteorder == 'little':
return b''.join(chars).decode('utf-16le')
return b''.join(chars).decode('utf-16be')
class c_utf16le_p(ctypes.c_char_p):
def __init__(self, value=None):
super(c_utf16le_p, self).__init__()
if value is not None:
self.value = value
@property
def value(self,
c_void_p=ctypes.c_void_p):
addr = c_void_p.from_buffer(self).value
return decode_utf16_from_address(addr, 'little')
@value.setter
def value(self, value,
c_char_p=ctypes.c_char_p):
value = value.encode('utf-16le') + b'\x00'
c_char_p.value.__set__(self, value)
@classmethod
def from_param(cls, obj):
if isinstance(obj, unicode):
obj = obj.encode('utf-16le') + b'\x00'
return super(c_utf16le_p, cls).from_param(obj)
@classmethod
def _check_retval_(cls, result):
return result.value
class UTF16LEField(object):
def __init__(self, name):
self.name = name
def __get__(self, obj, cls,
c_void_p=ctypes.c_void_p,
addressof=ctypes.addressof):
field_addr = addressof(obj) + getattr(cls, self.name).offset
addr = c_void_p.from_address(field_addr).value
return decode_utf16_from_address(addr, 'little')
def __set__(self, obj, value):
value = value.encode('utf-16le') + b'\x00'
setattr(obj, self.name, value)
Example:
if __name__ == '__main__':
class Test(ctypes.Structure):
_fields_ = (('x', ctypes.c_int),
('y', ctypes.c_void_p),
('_string', ctypes.c_char_p))
string = UTF16LEField('_string')
print('test 1: structure field')
t = Test()
t.string = u'eggs and spam'
print(t.string)
print('test 2: parameter and result')
result = None
@ctypes.CFUNCTYPE(c_utf16le_p, c_utf16le_p)
def testfun(string):
global result
print('parameter: %s' % string.value)
# callbacks leak memory except for simple return
# values such as an integer address, so return the
# address of a global variable.
result = c_utf16le_p(string.value + u' and eggs')
return ctypes.c_void_p.from_buffer(result).value
print('result: %s' % testfun(u'spam'))
Output:
test 1: structure field
eggs and spam
test 2: parameter and result
parameter: spam
result: spam and eggs
来源:https://stackoverflow.com/questions/35500018/how-to-work-with-utf-16-in-python-ctypes