C#/.NET - Custom Binary File Formats - Where to Start?

Question

I need to be able to store some data in a custom binary file format. I\'ve never designed my own file format before. It needs to be a friendly format for traveling between the C

Answer 1

I'll try to add some general hints for creating a portable binary file format.

Note that to invent a binary file format means to document, how the bits in it must go and what they mean. It's not coding, but documentation.

Now the hints:

1. Decide what to do with endianess. Good and simple way to go is to decide it once and forever. The choice would be preferably little endian when used on common PC (that is x86) to save conversions (performance).

2. Create header. Yes, it is good idea to always have a header. First bytes of the file should be able to tell you, what format you are messing with.

   • Start with magic to be able to recognize your format (ASCII string will do the trick)
   • Add version. Version of your file format will not hurt to add and it will allow you to do backward compatibility later.

3. Finally, add the data. Now, the format of the data will be specific and it will always be based on your exact needs. Basically, the data will be stored in a binary image of some data structure. The data structure is what you need to come up with.

If you need random access to your data by some sort of indices, B-Trees are way to go, while if you just need a lot of numbers to write them all and then read them all an "array" will do the trick.

Additionally, you might use a TLV (Type-Length-Value) concept for forward compatibility.