UnicodeDecodeError when reading CSV file in Pandas with Python

Question

I\'m running a program which is processing 30,000 similar files. A random number of them are stopping and producing this error...

File "C:\\Importer\\src         


        

Answer 1

Check the encoding before you pass to pandas. It will slow you down, but...

with open(path, 'r') as f:
    encoding = f.encoding 

df = pd.read_csv(path,sep=sep, encoding=encoding)

In python 3.7