UnicodeEncodeError: 'charmap' codec can't encode characters

Question

I\'m trying to scrape a website, but it gives me an error.

I\'m using the following code:

import urllib.request
from bs4 import BeautifulSoup

get =          


        

Answer 1

In Python 3.7, and running Windows 10 this worked (I am not sure whether it will work on other platforms and/or other versions of Python)

Replacing this line:

with open('filename', 'w') as f:

With this:

with open('filename', 'w', encoding='utf-8') as f:

The reason why it is working is because the encoding is changed to UTF-8 when using the file, so characters in UTF-8 are able to be converted to text, instead of returning an error when it encounters a UTF-8 character that is not suppord by the current encoding.

Answer 2

if you are using windows try to pass encoding='latin1', encoding='iso-8859-1' or encoding='cp1252' example:

csv_data = pd.read_csv(csvpath,encoding='iso-8859-1')
print(print(soup.encode('iso-8859-1')))