Bulletproof work with encoding in Python

Question

The question about unicode in Python2.

As I know about this I should always decode everything what I read from outside (files, net). decode

Answer 1

Try wrapping your functions in try:except: calls.

• Try decoding as utf-8:
• Catch exception if not utf-8:
• if exception raised, try next encoding:
• etc, etc...

Make it a function that returns str when (and if) it finds an encoding that wasn't excepted, and returns None or an empty str when it exhausts its list of encodings and the last exception is raised.

Like the others said, the encoding should be recorded somewhere, so check that first.

Not efficient, and frankly due to my skill level, may be way off, but to my newbie mind, it may alleviate some of the problems when dealing with unknown or undocumented encoding.