Is endian conversion required for wchar_t data?

Question

In C/C++, if a multi-byte wide character (wchar_t) value is transmitted from a big-endian system to a little-endian system (or vice-versa), will it come out the same value on the other side? Or will the bytes need to be swapped?

Answer 1

Yes you will need to swap them.
The bytes will be retrieved from the transport in the same order they were put in. Just at the other end the ordering of these bytes has a different meaning. So you need to convert them to the correct endian-ness (is that a word?).

The tried and true method is to convert to network byte order before transport. Then convert back to host specific byte order (from network byte order) on receipt.

A set of function to help with endian conversion:

ntohs   Convert a 16-bit quantity from network byte order to host byte order
ntohl   Convert a 32-bit quantity from network byte order to host byte order
htons   Convert a 16-bit quantity from host byte order to network byte order
htonl   Convert a 32-bit quantity from host byte order to network byte order

Just to add another note of caution.
Different systems use different size for wchar_t so do not assume sizeof(wchar_t) == 2.

Additionally each host may use a different representational format for wchar_t.
To help deal with this most systems convert the text to a known format for transport (UTF-8 or UTF-16 are good choices). The convert the text back to the host specific format at the other end.

You could look at IBM's icu this has all this functionality.

Answer 2

Endian conversion is not sufficient and as a consequence not needed. Sizeof(wchar_t) differs, and therefore the encoding too. Hence, you need to agree on an interchange format. The logical choice is UTF-8. But since UTF-8 is byte-oriented, you do not have endianness issues anymore.

Answer 3

Yes, you need to perform endian conversion. Define carefully your serialization format, i.e. the byte order of data that is transmitted over the network or stored into a disk file. Then, when sending data, convert from native to wire format (may or may not require byte swapping), and when receiving data, convert from wire to native format (again may or may not require byte swapping). You should pick a wire format that will be used by the majority of clients to minimize the average amount of byte swapping.