BeautifulSoup gives me unicode+html symbols, rather than straight up unicode. Is this a bug or misunderstanding?

Question

I\'m using BeautifulSoup to scrape a website. The website\'s page renders fine in my browser:

Oxfam International’s report entitled “Offside! http:

Answer 1

It's actually UTF-8 misencoded as CP1252:

>>> print u'Oxfam International\xe2€™s report entitled \xe2€œOffside!'.encode('cp1252').decode('utf8')
Oxfam International’s report entitled “Offside!