python-unicode

I'm trying to write out a csv file with Unicode characters, so I'm using the unicodecsv package. Unfortunately, I'm still getting UnicodeDecodeErrors: # -*- coding: utf-8 -*- import codecs import unicodecsv raw_contents = 'He observes an “Oversized Gorilla” near Ashford' encoded_contents = unicode(raw_contents, errors='replace') with codecs.open('test.csv', 'w', 'UTF-8') as f: w = unicodecsv.writer(f, encoding='UTF-8') w.writerow(["1", encoded_contents]) This is the traceback: Traceback (most recent call last): File "unicode_test.py", line 11, in <module> w.writerow(["1", encoded_contents])

I am trying this since morning. My sample.txt choice = \u9078\u629e Code: with open('sample.txt', encoding='utf-8') as f: for line in f: print(line) print("選択" in line) print(line.encode('utf-8').decode('utf-8')) print(line.encode().decode('utf-8')) print(line.encode('utf-8').decode()) print(line.encode().decode('unicode-escape').encode("latin-1").decode('utf-8')) # as suggested. out: choice = \u9078\u629e False choice = \u9078\u629e choice = \u9078\u629e choice = \u9078\u629e UnicodeEncodeError: 'latin-1' codec can't encode characters in position 9-10: ordinal not in range(256) When I do this

I was bitten by http://bugs.python.org/issue1681974 - quoting from there: mkdtemp fails on Windows if Windows user name has any non-ASCII characters, like ä or ö, in it. mkdtemp throws an encoding error. This seems to be because the default temp dir in Windows is "c:\documents and settings\<user name>\local settings\temp" The workaround the OP used is: try: # workaround for http://bugs.python.org/issue1681974 return tempfile.mkdtemp(prefix=prefix) except UnicodeDecodeError: tempdir = unicode(tempfile.gettempdir(), 'mbcs') return tempfile.mkdtemp(prefix=prefix, dir=tempdir) I have 2 questions:

With the following code lst = [u'\u5de5', u'\u5de5'] msg = repr(lst).decode('unicode-escape') print msg I got [u'工', u'工'] How can I remove the leading u so that the content of msg is: ['工', '工'] >>> import sys >>> lst = [u'\u5de5', u'\u5de5'] >>> msg = repr([x.encode(sys.stdout.encoding) for x in lst]).decode('string-escape') >>> print msg ['工', '工'] 来源： https://stackoverflow.com/questions/22745876/python-print-unicode-list

I have Python 2.7.1 on a Simplified-Chinese version of Windows XP, and I have a program like this(windows_prn_utf8.py): #!/usr/bin/env python # -*- coding: utf8 -*- print unicode('\xE7\x94\xB5', 'utf8') If I run it on Windows CMD console, it output the right Chinese character '电' ; however, if I try to redirect the command output to a file. I got error. D:\Temp>windows_prn_utf8.py > 1.txt Traceback (most recent call last): File "D:\Temp\windows_prn_utf8.py", line 4, in <module> print unicode('\xE7\x94\xB5', 'utf8') UnicodeEncodeError: 'ascii' codec can't encode character u'\u7535' in position

I'm a python beginner. I'm trying to crawl google play store and export to csv file. But I got a error message. UnicodeEncodeError: 'cp949' codec can't encode character '\u20a9' in position 90: illegal multibyte sequence Here is my source code. When I command print, it works. But it shows error message when exporting to csv file please help me from bs4 import BeautifulSoup import urllib.request import urllib.parse import codecs import json import pickle from datetime import datetime import sys import csv import os req = 'https://play.google.com/store/search?q=hana&c=apps&num=300' response =