Pickle alternatives

Question

I am trying to serialize a large (~10**6 rows, each with ~20 values) list, to be used later by myself (so pickle\'s lack of safety isn\'t a concern).

Each row of the lis

Answer 1

• Protocol Buffer - e.g. used in Caffe; maintains type information, but you have to put quite much effort in it compared to pickle
• MessagePack: See python package - supports streaming (source)
• BSON: see python package docs