pympler asizeof vs sys.getsizeof

Question

I have a pickled filed. Its size is 9.3MB.

-rw-r--r-- 1 ankit ankit 9.3M Jan  7 17:43 agg_397127.pkl

I load it in python using cPickle. I

Answer 1

sys.getsizeof() returns the size of the object passed to it - which is a dictionary with one entry, in your example. It does NOT include the size of the complex class instance referred to by the dictionary, nor any of the objects referred to by that instance. ANY dictionary with only a few entries (up to 5, on my Python version) would return exactly the same number.

The assizeof module you're using attempts to recursively add up the sizes of all these referred objects. It doesn't seem to have done a very good job in this case, considering the huge discrepancy between the size returned and the pickle size (but note that these numbers would never be exactly equal, since the format of a pickle on disk is necessarily different than the format of the actual objects in memory).