marshal dumps faster, cPickle loads faster

Question

I\'m implementing a program that needs to serialize and deserialize large objects, so I was making some tests with pickle, cPickle and marsha

Answer 1

As you can see, the output produced by cPickle.dump has about 1/4 of the length of the output produced by marshal.dump. This means that cPickle must use a more complicated algorithm to dump the data as unneeded things are removed. When loading the dumped list, marshal has to work through much more data while cPickle can process its data quickly as there is less data that has to be analysed.

Regarding the fact that marshal might be incompatible to other versions of Python, you should generally use cPickle:

"This is not a general “persistence” module. For general persistence and transfer of Python objects through RPC calls, see the modules pickle and shelve. The marshal module exists mainly to support reading and writing the “pseudo-compiled” code for Python modules of .pyc files. Therefore, the Python maintainers reserve the right to modify the marshal format in backward incompatible ways should the need arise. If you’re serializing and de-serializing Python objects, use the pickle module instead – the performance is comparable, version independence is guaranteed, and pickle supports a substantially wider range of objects than marshal." (the python docs about marshal)