How to serialize/deserialize large list of items with protobuf-net

Question

I have a list of about 500 million items. I am able to serialize this into a file with protobuf-net file if I serialize individual items, not a list -- I cannot collect the item

Answer 1

May be I am too late on this... but just to add to what Marc already said.

As you use Serializer.Serialize(output, price); protobuf treat consecutive messages as part of a (same)single object. So when you use Deserialize using

while ((price = Serializer.Deserialize(input)) != null)

you will get all the records back. Hence you will see only the last Price record.

To do what you want to do, change the serialization code to:

Serializer.SerializeWithLengthPrefix(output, price, PrefixStyle.Base128, 1);

and

while ((price = Serializer.DeserializeWithLengthPrefix(input, PrefixStyle.Base128, 1)) != null)