Supporting byte ordering in Linux user space

问题

I'm writing a program on Linux in C to analyze core files produced from an embedded system. The core files might be little endian (ARM) or big endian (MIPS), and the program to analyze them might be running on a little endian host (x86) or big endian (PowerPC).

By looking at the headers I know whether the core is LE or BE. I'd rather my program not need to know whether the host it runs on is little or big endian, I'd like to use an API to handle it for me. If there is no better option, I guess I'll start relying on #ifdef __BIG_ENDIAN__.

In the Linux kernel we have cpu_to_le32 et al to convert from native byte ordering to little endian, etc. In user space there is htonl et al, which convert from native to big endian but no equivalent for native to little endian that I can find.

Can anyone suggest a suitable API for user space?

Edit: Just to be clear, I'm looking for an API which already knows whether my CPU is big or little endian and swaps byes accordingly. I don't want to have to litter my code with #ifdefs for this. I'm not just looking for code snippets to swap bytes; thank you for those, but that wasn't the point.

回答1:

Based on what you're actually trying to do (read ELF core dump files without having to worry about endian issues), I believe using libelf, available here with a nice tutorial here, would be a good choice.

This library works transparently with big- and little-endian ELF files and runs just fine under Linux despite its FreeBSD origins (the usual "./configure" and "make" sequence is all you'll need to build it). For grins I tried the "reading a program header table" example (with minor modifications to get it to build) on an x86 core file as well as a MIPS big-endian core file, it appears to "just work".

回答2:

#include <arpa/inet.h> is nice and portable, but only guarantees {ntoh,hton}{s,l}. If you need conversions on 64-bit values, or endian flipping on big-endian (where ntoh and hton do nothing), it won't be enough.

On Linux (glibc), #include <endian.h> provides the following, defined as appropriate for the current machine.

htobe16  be16toh    htole16  le16toh
htobe32  be32toh    htole32  le32toh
htobe64  be64toh    htole64  le64toh

On *BSD, #include <sys/endian.h> provides these same macros.

回答3:

If you have access to the neon coprocessor and the memory is contiguous (example a video frame) you could execute swap16 on the frame using the q registers (128bytes) in this way; of course, you have to watch for alignment issues

void swap16(void *__restrict src16)
{
    const void *start = src16;
    const void *end = src16 + FRAME_SIZE;
    asm volatile (
        "1: pld     [%0, #0x100]\n"
        "vld2.8         {q0,q1}, [%0]\n"
        "vmov           q2,q0\n"
        "vst2.8         {q1,q2}, [%0]!\n"
        "cmp            %1,%0\n"
        "bne            1b\n"
        : /* empty output operands */
        : "r" (start), "r" (end)
        : "cc", "memory"
        );
}

回答4:

If you treat the file as a byte array, then you control which order you pick the bytes out, and the endian-ness of your CPU actually ends up not mattering.

This is also highly useful in terms of dealing with alignment problems. Your core dump might concievably have unaligned references in it. I know that's highly unlikely, but it could be due to corruption as well. This is another problem that is avoided by treating the file as a byte array.

I used this approach in implementing jhead.

回答5:

You can use the standard network bytswapping functions in apa/inet.h:

#include <arpa/inet.h>
uint32_t htonl(uint32_t hostlong); // Host to network
uint16_t htons(uint16_t hostshort); // Host to network
uint32_t ntohl(uint32_t netlong); // Network to host
uint16_t ntohs(uint16_t netshort); // Network to host

Network byte order is big-endian. So, these functions mean:

hton*: Host endian to big endian
ntoh*: Big endian to host endian

Hope this helps.

回答6:

Take a look at the kernel-provided headers in /usr/include/linux/byteorder/ such as __cpu_to_be32() and __be32_to_cpu()

Take also a look at the /usr/include/linux/types.h file where you can define types as explicit big/little endian plain integers, which help a lot since any mismatch will be detected at compile time.

回答7:

Why do you need an API to do it? Simply write your own function to call htonl() (or whatever produces BE) then simply reverse the bytes. That doesn't sound so hard.

Something like:

union {
    struct {
        unsigned char c0;
        unsigned char c1;
        unsigned char c2;
        unsigned char c3;
    } ch;
    uint32_t ui;
} u;
unsigned char t;

u.ui = htonl (hostlong);
t = u.ch.c0; u.ch.c0 = u.ch.c3 ; u.ch.c3 = t;
t = u.ch.c1; u.ch.c1 = u.ch.c2 ; u.ch.c2 = t;

回答8:

Given that switching endian-ess is easy, I always ended up using custom code like that, keeping a strict rule about what representation I use in the code, and handling endianity at the end points (input and output).

回答9:

You could just write your own (these are based on Apple's routines):

static inline uint16_t Swap16(uint16_t x)
{
    return ( (x << 8) | (x >> 8) );
}

static inline uint32_t Swap32(uint32_t x)
{
    return ( (((x ^ (x >> 16 | (x << 16))) & 0xff00ffff) >> 8) ^ (x >> 8 | data << 24) );
}

Then you can define conditional macros:

#ifdef __BIG_ENDIAN__
# define htols(x) Swap16(x)
# define htoll(x) Swap32(x)
#else
# define htols(x) (x)
# define htoll(x) (x)
#endif

If you're happy with Intel assembler code, you can even do this:

// Swap16 is unchanged

static inline uint32_t Swap32(uint32_t x)
{
    __asm__ ("bswap %0" : "+r" (x));
    return ( x );
}
#ifdef __i386__
static inline uint64_t Swap64(uint64_t x)
{
    __asm__ ("bswap  %%eax\n\t"
             "bswap  %%edx\n\t"
             "xchgl  %%eax, %%edx"
             : "+A" (x));
    return ( x );
}
#elif defined(__x86_64__)
static inline uint64_t Swap64( uint64_t x )
{
    __asm__ ("bswap  %0" : "+r" (x));
    return ( x );
}
#endif

来源：https://stackoverflow.com/questions/874617/supporting-byte-ordering-in-linux-user-space

标签

Linux

endianness