Python (nltk) - UnicodeDecodeError: 'ascii' codec can't decode byte

Question

I\'m new to NLTK. I\'m getting this error and I\'ve searched around for encoding/decoding and specifically the UnicodeDecodeError but this error seems specific to the NLTK s

Answer 1

try this... NLTK 3.0.1 with Python 2.7.x

import io
f = io.open(txtFile, 'rU', encoding='utf-8')

Answer 2

I had the same problem with you. I use Python 3.4 in Windows 7.

I had installed the "nltk-3.0.0.win32.exe" (from here). But when i installed the "nltk-3.0a4.win32.exe" (from here), my problem with nltk.pos_tag was solved. Check it.

EDIT: If the second link doesn't work, you can look here.

Answer 3

Try using the module "textclean"

>>> pip install textclean

Python code

from textclean.textclean import textclean
text = textclean.clean("John's big idea isn't all that bad.")
print pos_tag(word_tokenize(text))

Answer 4

Duplicate: NLTK 3 POS_TAG throws UnicodeDecodeError

Long story short: NLTK isn't compatible with Python 3. You have to use NLTK 3 which sounds a bit experimental at this point.