iconv

嵌入式开发中中文的识别、显示一直是一个比较让人头疼的问题，这是由于嵌入式系统的精简要求所导致的。我在前一个项目中用Qt做终端的显示界面时，就被中文字符的显示搞得焦头烂额，最后还是在网上下了一个文泉驿的中文字体库才解决问题，不过并不令人满意，主要是字体库太大了，影响了性能。今天在项目中又遇到了中文字符的编码转换和显示问题，趁机学习总结一下。 中文编码与点阵字库 国际标准有UTF-8、UTF-16等，国家标准有GB2312、GB18030、GBK等，至于这些标准的编码实现就不在这里介绍了。编码标准实现中文字符在计算机内部的机器表示，但是怎么将这些内部表示在显示设备上显示出来，则不是编码标准的范畴了，这就需要中文字库的作用。中文字库包含已经做好的显示“模板”(字模)，通过中文的编码值来映射汉字在字库中的字模索引，从而使用字模来实现中文显示。 但是大部分的中文字库都是映射国家标准的，如GB2312的HZK16点阵字库。不知道文泉驿字体库是否是直接映射UTF-8，还是也是GB2312。因此，当在程序中涉及两种不同的中文编码方式时，就需要进行编码方式的转换，今天我遇到的就是要实现中文字符UTF-8到GB2312的编码转换。 HZK16是符合GB2312标准16*16的点阵宋体字库，支持的汉字有6763个，符号682个。其中一级汉字有 3755个，按声序排列，二级汉字有3008个，按偏旁部首排列

Error while installing iconv on windows by ruby2.0.0 Error while installing iconv on windows by ruby2.0.0 I'm trying to install the iconv , but when I run the following command: gem install iconv my result: Temporarily enhancing Path to include DevKit... Building native extensions. This could take a while... Error : Error installing iconv: ERROR: Failed to build gem native extension. C:/Ruby/Ruby200/bin/ruby.exe -r ./siteconf20150217-3212-i8hzmh.rb extconf.rb checking for rb_enc_get() in ruby/encoding.h... yes checking for rb_sys_fail_str() in ruby.h... yes checking for iconv() in iconv.h...

Encoding convert from windows-1255 to utf-8 was asked before I know, but I'm still getting different results and I can't solve it. The first issue is "does php iconv() or mb_convert_encoding() support windows-1255????" While testing, it returns several outputs (playing with the //ignore & //translate) but its not working well at all. I was looking at mb_list_encodings() output and it doesn't include window-1255... playing and testing mb_detect_encoding() with an windows-1255 input (crawled from the net) doesn't return the good charset... You should be able to just use strtr with an associative

I'm trying to get git to recognize UTF-16 as text to allow me to diff and patch as text natively, but I'm having trouble getting the textconv parameter to work. I can manually call iconv -f utf-16 -t utf-8 some-utf-16-file.rc and everything is fine. But if I configure my .gitconfig as follows [diff "utf16"] textconv = "iconv -f utf-16le -t utf-8" and my .gitattributes: # Custom for MFC *.rc text eol=crlf diff=utf16 However, if I then if I run git diff , the following is displayed: iconv: C:/Users/Mahmoud/AppData/Local/Temp/IjLBZ8_OemKey.rc:104:1: incomplete character or shift sequence With

Inside a React Native method I'm fetching a xml encoded in ISO-8859-1. As long as the fetching is completed I'm trying to convert it to UTF-8. Here the code: const iconv = require('iconv-lite'); fetch('http://www.band.uol.com.br/rss/colunista_64.xml', { headers: { "Content-type": "text/xml; charset=ISO-8859-1" } }) .then(res=>res.text()}) .then(text => { const decodedText = iconv.decode(Buffer.from(text, 'latin1'), 'latin1') , output = iconv.encode(decodedText, 'utf8') console.log(output.toString()) }) The problem is: all especial characters of the body is being replaced by "¿½" For the

I've tried to change filenames using following script: find dir/ -type f -exec mv {} $(echo {} | iconv -f UTF8 -t ASCII//TRANSLIT ) \; Why doesn't it work? I mean when I have a file with character like 'ą' it should convert it to 'a'. $ echo ążźćó | iconv -f UTF8 -t ASCII//TRANSLIT azzco why does it not work in find -exec? $ find dir/ -type f -exec mv {} $(echo {} | iconv -f UTF8 -t ASCII//TRANSLIT ) \; mv: `dir/zią' and `dir/zią' are the same file I get the same results using xargs: $ find dir/ -type f | xargs -I{} echo {} | iconv -f UTF8 -t ASCII//TRANSLIT dir/zia but: $ find dir/ -type f |