Fastest way to store large files in Python

Question

I recently asked a question regarding how to save large python objects to file. I had previously run into problems converting massive Python dictionaries into string and writing

Answer 1

faster, or even possible, to zip this pickle file prior to [writing]

Of course it's possible, but there's no reason to try to make an explicit zipped copy in memory (it might not fit!) before writing it, when you can automatically cause it to be zipped as it is written, with built-in standard library functionality ;)

See http://docs.python.org/library/gzip.html . Basically, you create a special kind of stream with

gzip.GzipFile("output file name", "wb")

and then use it exactly like an ordinary file created with open(...) (or file(...) for that matter).