Performance reading large SPSS file in pandas dataframe on Windows 7 (x64)

Question

I have a large SPSS-file (containing a little over 1 million records, with a little under 150 columns) that I want to convert to a Pandas DataFrame.

It takes a few minut

Answer 1

You can use rawMode=True to speed up things a bit, as in:

raw_data = spss.SavReader('largefile.sav', returnHeader=True, rawMode=True)

This way, datetime variables (if any) won't be converted to ISO-strings, and SPSS $sysmis values won't be converted to None, and a few other things.