How do you serialize an object in C++?

Question

I have a small hierarchy of objects that I need to serialize and transmit via a socket connection. I need to both serialize the object, then deserialize it based on what ty

Answer 1

Talking about serialization, the boost serialization API comes to my mind. As for transmitting the serialized data over the net, I'd either use Berkeley sockets or the asio library.

Edit:
If you want to serialize your objects to a byte array, you can use the boost serializer in the following way (taken from the tutorial site):

#include <boost/archive/binary_oarchive.hpp>
#include <boost/archive/binary_iarchive.hpp>
class gps_position
{
private:
    friend class boost::serialization::access;
    template<class Archive>
    void serialize(Archive & ar, const unsigned int version)
    {
        ar & degrees;
        ar & minutes;
        ar & seconds;
    }
    int degrees;
    int minutes;
    float seconds;

public:
    gps_position(){};
    gps_position(int d, int m, float s) :
    degrees(d), minutes(m), seconds(s)
    {}
};

Actual serialization is then pretty easy:

#include <fstream>
std::ofstream ofs("filename.dat", std::ios::binary);

    // create class instance
    const gps_position g(35, 59, 24.567f);

    // save data to archive
    {
        boost::archive::binary_oarchive oa(ofs);
        // write class instance to archive
        oa << g;
        // archive and stream closed when destructors are called
    }

Deserialization works in an analogous manner.

There are also mechanisms which let you handle serialization of pointers (complex data structures like tress etc are no problem), derived classes and you can choose between binary and text serialization. Besides all STL containers are supported out of the box.

Answer 2

By way of learning I wrote a simple C++11 serializer. I had tried various of the other more heavyweight offerings, but wanted something that I could actually understand when it went wrong or failed to compile with the latest g++ (which happened for me with Cereal; a really nice library but complex and I could not grok the errors the compiler threw up on upgrade.) Anyway, it's header only and handles POD types, containers, maps etc... No versioning and it will only load files from the same arch it was saved in.

https://github.com/goblinhack/simple-c-plus-plus-serializer

Example usage:

#include "c_plus_plus_serializer.h"

static void serialize (std::ofstream out)
{
    char a = 42;
    unsigned short b = 65535;
    int c = 123456;
    float d = std::numeric_limits<float>::max();
    double e = std::numeric_limits<double>::max();
    std::string f("hello");

    out << bits(a) << bits(b) << bits(c) << bits(d);
    out << bits(e) << bits(f);
}

static void deserialize (std::ifstream in)
{
    char a;
    unsigned short b;
    int c;
    float d;
    double e;
    std::string f;

    in >> bits(a) >> bits(b) >> bits(c) >> bits(d);
    in >> bits(e) >> bits(f);
}

Answer 3

In some cases, when dealing with simple types, you can do:

object o;
socket.write(&o, sizeof(o));

That's ok as a proof-of-concept or first-draft, so other members of your team can keep working on other parts.

But sooner or later, usually sooner, this will get you hurt!

You run into issues with:

• Virtual pointer tables will be corrupted.
• Pointers (to data/members/functions) will be corrupted.
• Differences in padding/alignment on different machines.
• Big/Little-Endian byte ordering issues.
• Variations in the implementation of float/double.

(Plus you need to know what you are unpacking into on the receiving side.)

You can improve upon this by developing your own marshalling/unmarshalling methods for every class. (Ideally virtual, so they can be extended in subclasses.) A few simple macros will let you to write out different basic types quite quickly in a big/little-endian-neutral order.

But that sort of grunt work is much better, and more easily, handled via boost's serialization library.

Answer 4

There is a generic pattern you can use to serialize objects. The fundemental primitive is these two functions you can read and write from iterators:

template <class OutputCharIterator>
void putByte(char byte, OutputCharIterator &&it)
{
    *it = byte;
    ++it;
}


template <class InputCharIterator>
char getByte(InputCharIterator &&it, InputCharIterator &&end)
{
    if (it == end)
    {
        throw std::runtime_error{"Unexpected end of stream."};
    }

    char byte = *it;
    ++it;
    return byte;
}

Then serialization and deserialization functions follow the pattern:

template <class OutputCharIterator>
void serialize(const YourType &obj, OutputCharIterator &&it)
{
    // Call putbyte or other serialize overloads.
}

template <class InputCharIterator>
void deserialize(YourType &obj, InputCharIterator &&it, InputCharIterator &&end)
{
    // Call getByte or other deserialize overloads.
}

For classes you can use the friend function pattern to allow the overload to be found using ADL:

class Foo
{
    int internal1, internal2;

    // So it can be found using ADL and it accesses private parts.
    template <class OutputCharIterator>
    friend void serialize(const Foo &obj, OutputCharIterator &&it)
    {
        // Call putByte or other serialize overloads.
    }

    // Deserialize similar.
};

The in your program you can serialize and object into a file like this:

std::ofstream file("savestate.bin");
serialize(yourObject, std::ostreambuf_iterator<char>(file));

Then read:

std::ifstream file("savestate.bin");
deserialize(yourObject, std::istreamBuf_iterator<char>(file), std::istreamBuf_iterator<char>());

---

My old answer here:

Serialization means turning your object into binary data. While deserialization means recreating an object from the data.

When serializing you are pushing bytes into an uint8_t vector. When unserializing you are reading bytes from an uint8_t vector.

There are certainly patterns you can employ when serializing stuff.

Each serializable class should have a serialize(std::vector<uint8_t> &binaryData) or similar signatured function that will write its binary representation into the provided vector. Then this function may pass this vector down to it's member's serializing functions so they can write their stuff into it too.

Since the data representation can be different on different architectures. You need to find out a scheme how to represent the data.

Let's start from the basics:

Serializing integer data

Just write the bytes in little endian order. Or use varint representation if size matters.

Serialization in little endian order:

data.push_back(integer32 & 0xFF);
data.push_back((integer32 >> 8) & 0xFF);
data.push_back((integer32 >> 16) & 0xFF);
data.push_back((integer32 >> 24) & 0xFF);

Deserialization from little endian order:

integer32 = data[0] | (data[1] << 8) | (data[2] << 16) | (data[3] << 24);

Serializing floating point data

As far as I know the IEEE 754 has a monopoly here. I don't know of any mainstream architecture that would use something else for floats. The only thing that can be different is the byte order. Some architectures use little endian, others use big endian byte order. This means you need to be careful which order to you loud up the bytes on the receiving end. Another difference can be handling of the denormal and infinity and NAN values. But as long as you avoid these values you should be OK.

Serialization:

uint8_t mem[8];
memcpy(mem, doubleValue, 8);
data.push_back(mem[0]);
data.push_back(mem[1]);
...

Deserialization is doing it backward. Mind the byte order of your architecture!

Serializing strings

First you need to agree on an encoding. UTF-8 is common. Then store it as a length prefixed manner: first you store the length of the string using a method I mentioned above, then write the string byte-by-byte.

Serializing arrays.

They are the same as a strings. You first serialize an integer representing the size of the array then serialize each object in it.

Serializing whole objects

As I said before they should have a serialize method that add content to a vector. To unserialize an object, it should have a constructor that takes byte stream. It can be an istream but in the simplest case it can be just a reference uint8_t pointer. The constructor reads the bytes it wants from the stream and sets up the fields in the object. If the system is well designed and serialize the fields in object field order, you can just pass the stream to the field's constructors in an initializer list and have them deserialized in the right order.

Serializing object graphs

First you need to make sure if these objects are really something you want to serialize. You don't need to serialize them if instances of these objects present on the destination.

Now you found out you need to serialize that object pointed by a pointer. The problem of pointers that they are valid only the in the program that uses them. You cannot serialize pointer, you should stop using them in objects. Instead create object pools. This object pool is basically a dynamic array which contains "boxes". These boxes have a reference count. Non-zero reference count indicates a live object, zero indicates an empty slot. Then you create smart pointer akin to the shared_ptr that doesn't store the pointer to the object, but the index in the array. You also need to agree on an index that denotes the null pointer, eg. -1.

Basically what we did here is replaced the pointers with array indexes. Now when serializing you can serialize this array index as usual. You don't need to worry about where does the object will be in memory on the destination system. Just make sure they have the same object pool too.

So we need to serialize the object pools. But which ones? Well when you serialize an object graph you are not serializing just an object, you are serializing an entire system. This means the serialization of the system shouldn't start from parts of the system. Those objects shouldn't worry about the rest of the system, they only need to serialize the array indexes and that's it. You should have a system serializer routine that orchestrates the serialization of the system and walks through the relevant object pools and serialize all of them.

On the receiving end all the arrays an the objects within are deserialized, recreating the desired object graph.

Serializing function pointers

Don't store pointers in the object. Have a static array which contains the pointers to these functions and store the index in the object.

Since both programs have this table compiled into themshelves, using just the index should work.

Serializing polymorphic types

Since I said you should avoid pointers in serializable types and you should use array indexes instead, polymorphism just cannot work, because it requires pointers.

You need to work this around with type tags and unions.

Versioning

On top of all the above. You might want different versions of the software interoperate.

In this case each object should write a version number at the beginning of their serialization to indicate version.

When loading up the object at the other side the, newer objects maybe able to handle the older representations but the older ones cannot handle the newer so they should throw an exception about this.

Each time a something changes, you should bump the version number.

---

So to wrap this up, serialization can be complex. But fortunately you don't need to serialize everything in your program, most often only the protocol messages are serialized, which are often plain old structs. So you don't need the complex tricks I mentioned above too often.