Portability of binary serialization of double/float type in C++

Question

The C++ standard does not discuss the underlying layout of float and double types, only the range of values they should represent. (This is also true for signed types, is i

Answer 1

Create an appropriate serializer/de-serializer interface for writing/reading this.

The interface can then have several implementations and you can test your options.

As said before, obvious options would be:

• IEEE754 which writes / reads the binary chunk if directly supported by the architecture or parses it if not supported by the architecture
• Text: always needs to parse.
• Whatever you else you can think of.

Just remember - once you have this layer, you can always start with IEEE754 if you only support platforms that use this format internally. This way you'll have the additional effort only when you need to support a different platform! Don't do work you don't have to.

Answer 2

I think the answer "depends" on what your particular application and it's perfomance profile is.

Let's say you have a low-latency market data environment, then using strings is frankly daft. If the information you are conveying is prices, then doubles (and binary representation of them) really are tricky to work with. Where as, if you don't really care about performance, and what you want is visibility (storage, transmission), then strings are an ideal candidate.

I would actually opt for integral mantissa/exponent representation of floats/doubles - i.e. at the earliest opportunity, convert the float/double to a pair of integers and then transmit that. You then only have to worry about the portability of integers and well, various routines (such as the hton() routines to handle conversions for you). Also store everything in your most prevalent platform's endianess (for example if you're only using linux, then what's the point of storing stuff in big endian?)

Answer 3

The SQLite4 uses a new format to store doubles and floats

• It works reliably and consistently even on platforms that lack support for IEEE 754 binary64 floating point numbers.
• Currency computations can normally be done exactly and without rounding.
• Any signed or unsigned 64-bit integer can be represented exactly.
• The floating point range and accuracy exceed that of IEEE 754 binary64 floating point numbers.
• Positive and negative infinity and NaN (Not-a-Number) have well-defined representations.

Sources:

https://sqlite.org/src4/doc/trunk/www/design.wiki

https://sqlite.org/src4/doc/trunk/www/decimal.wiki

Answer 4

You should convert them to a format you will always be able to use in order to recreate your floats/doubles.

This could use a string representation or, if you need something that takes less space, represent your number in ieee754 (or any other format you choose) and then parse it as you would do with a string.

Answer 5

Brian "Beej Jorgensen" Hall gives in his Guide to Network Programming some code to pack float (resp. double) to uint32_t (resp. uint64_t) to be able to safely transmit it over the network between two machine that may not both agree to their representation. It has some limitation, mainly it does not support NaN and infinity.

Here is his packing function:

#define pack754_32(f) (pack754((f), 32, 8))
#define pack754_64(f) (pack754((f), 64, 11))

uint64_t pack754(long double f, unsigned bits, unsigned expbits)
{
    long double fnorm;
    int shift;
    long long sign, exp, significand;
    unsigned significandbits = bits - expbits - 1; // -1 for sign bit

    if (f == 0.0) return 0; // get this special case out of the way

    // check sign and begin normalization
    if (f < 0) { sign = 1; fnorm = -f; }
    else { sign = 0; fnorm = f; }

    // get the normalized form of f and track the exponent
    shift = 0;
    while(fnorm >= 2.0) { fnorm /= 2.0; shift++; }
    while(fnorm < 1.0) { fnorm *= 2.0; shift--; }
    fnorm = fnorm - 1.0;

    // calculate the binary form (non-float) of the significand data
    significand = fnorm * ((1LL<<significandbits) + 0.5f);

    // get the biased exponent
    exp = shift + ((1<<(expbits-1)) - 1); // shift + bias

    // return the final answer
    return (sign<<(bits-1)) | (exp<<(bits-expbits-1)) | significand;
}

Answer 6

Just write the binary IEEE754 representation to disk, and document this as your storage format (along with is endianness). Then it's up to the implementation to convert this to its internal representation if necessary.