Python pickle not one-to-one: different pickles give same object

Question

Can someone explain this?

pickle.loads(b\'\\x80\\x03X\\x01\\x00\\x00\\x00.q\\x00h\\x00\\x86q\\x01.\') == pickle.loads(b\'\\x80\\x03X\\x01\\x00\\x00\\x00.q\\x00X\         


        

Answer 1

Pickles aren't unique; the pickle format is actually a tiny little programming language, and different programs (pickles) can produce the same output (unpickled object). From the docs:

Since the pickle data format is actually a tiny stack-oriented programming language, and some freedom is taken in the encodings of certain objects, it is possible that the two modules [pickle and cPickle] produce different data streams for the same input objects. However it is guaranteed that they will always be able to read each other’s data streams.

There's even a pickletools.optimize function that will take a pickle and output a better pickle. You're going to need to redesign your program.