Unpickling dictionary that holds pandas dataframes throws AttributeError: 'Dataframe' object has no attribute '_data'

Question

I have a class that performs analyses and attaches the results, which are pandas dataframes, as object attributes:

>>> print(test.image.locate_DF)
           


        

Answer 1

Maybe the problem has been solved.
Emmm, but I still want to add some comments.

I save the pkl file on the server, but when I load it on my MAC, it crashed, showing 'Dataframe' object has no attribute '_data'

Finally, I found that pandas on my Mac is 1.0.5 but 1.1.5 on the server. When I updated it to the latest, it just worked.

Answer 2

I had the same problem. I generated a Pandas dataframe in an environment with Pandas 1.1.1 and saved it to a pickle file.

with open('file.pkl', 'wb') as f:
    pickle.dump(data_frame_object, f)

After unpickling it in another session and printing the dataframe I got the same error. Some testing in different environments showed the following pattern:

• environment with Pandas >= 1.1.0: works
• environment with Pandas == 1.0.5: error message as above
• environment with Pandas == 1.0.3: Kernel crashes

I got the same error using the HDF5 format so it seems to be a compatibility issue with the dataframe and different Pandas versions.

Updating Pandas to 1.1.1 in the affected environments solved the issue for me.

Answer 3

After a long and painful process of cross-checking module versions, I found out that this error was caused due to an update in the pandas version. My mac still ran pandas 1.0.5, whereas the hpc runs pandas 1.1.0. Apparently, there is a mismatch between the two (unsure whether it's just after pickling or also for other file formats used to save).