Python - Unicode & double backslashes

Question

I scrapped a webpage with BeautifulSoup. I got great output except parts of the list look like this after getting the text:

list = [u\'that\\\\u2019s\', u\'it\\         


        

Answer 1

the problem here is that the site ended up double encoding those unicode arguments, just do the following:

ls = [u'that\\u2019s', u'it\\u2019ll', u'It\\u2019s', u'don\\u2019t', u'That\\u2019s', u'we\\u2019re', u'\\u2013']

ls = map(lambda x: x.decode('unicode-escape'), ls)

now you have a list with properly unicode encoded strings:

for a in ls:
   print a