Compress Python Object in Memory

Question

Most tutorials on compressing a file in Python involve immediately writing that file to disk with no intervening compressed python object. I want to know how to pickle and then

Answer 1

I use this to save memory in one place:

import cPickle
import zlib

# Compress:
compressed = zlib.compress(cPickle.dumps(obj))

# Get it back:
obj = cPickle.loads(zlib.decompress(compressed))

If obj has references to a number of small objects, this can reduce the amount of memory used by a lot. A lot of small objects in Python add up because of per-object memory overhead as well as memory fragmentation.