memory alignment within gcc structs

Question

I am porting an application to an ARM platform in C, the application also runs on an x86 processor, and must be backward compatible.

I am now having some issues with

Answer 1

What is your true goal?

If it's to deal with data that's in a file or on the wire in a particular format what you should do is write up some marshaling/serialization routines that move the data between the compiler struct that represents how you want to deal with the data inside the program and a char array that deals with how the data looks on the wire/file.

Then all that needs to be dealt with carefully and possibly have platform specific code is the marshaling routines. And you can write some nice-n-nasty unit tests to ensure that the marshaled data gets to and from the struct properly no matter what platform you might have to port to today and in the future.

Answer 2

I've been moving structures back and forth from Linux, Windows, Mac, C, Swift, Assembly, etc.

The problem is NOT that it can't be done, the problem is that you can't be lazy and must understand your tools.

I don't see why you can't use:

typedef struct  
{  
 unsigned int code;  
 unsigned int length;  
 unsigned int seq;  
 unsigned int request;  
 unsigned char nonce[16];  
 unsigned short  crc;  
} __attribute__((packed)) CHALLENGE;

You can use it and it doesn't require any special or clever code. I write a LOT of code that communicates to ARM. Structures are what make things work. __attribute__ ((packed)) is my friend.

The odds of being in a "world of hurt" are nil if you understand what is going on with both.

Finally, I can't for the life make out how you get 42 or 44. Int is either 4 or 8 bytes (depending on the compiler). That puts the number at either 16+16+2=34 or 32+16+2=50 -- assuming it is truly packed.

As I say, knowing your tools is part of your problem.

Answer 3

As I am using linux, I have found that by echo 3 > /proc/cpu/alignment it will issue me with a warning, and fix the alignment issue. This is a work around but it is very helpful with locating where the structures are failing to be misaligned.

Answer 4

This is one reason why reading whole structs instead of memberwise fails, and should be avoided.

In this case, packing plus aligning at 4 means there will be two bytes of padding. This happens because the size must be compatible for storing the type in an array with all items still aligned at 4.

I imagine you have something like:

read(fd, &obj, sizeof obj)

Because you don't want to read those 2 padding bytes which belong to different data, you have to specify the size explicitly:

read(fd, &obj, 42)

Which you can keep maintainable:

typedef struct {
  //...
  enum { read_size = 42 };
} __attribute__((aligned(4),packed)) CHALLENGE;

// ...

read(fd, &obj, obj.read_size)

Or, if you can't use some features of C++ in your C:

typedef struct {
  //...
} __attribute__((aligned(4),packed)) CHALLENGE;
enum { CHALLENGE_read_size = 42 };

// ...

read(fd, &obj, CHALLENGE_read_size)

At the next refactoring opportunity, I would strongly suggest you start reading each member individually, which can easily be encapsulated within a function.

Answer 5

I would guess that the problem is that 42 isn't divisible by 4, and so they get out of alignment if you put several of these structs back to back (e.g. allocate memory for several of them, determining the size with sizeof). Having the size as 44 forces the alignment in these cases as you requested. However, if the internal offset of each struct member remains the same, you can treat the 44 byte struct as though it was 42 bytes (as long as you take care to align any following data at the correct boundary).

One trick to try might be putting both of these structs inside a single union type and only use 42-byte version from within each such union.

Answer 6

If you're depending on sizeof(yourstruct) being 42 bytes, you're about to be bitten by a world of non-portable assumptions. You haven't said what this is for, but it seems likely that the endianness of the struct contents matters as well, so you may also have a mismatch with the x86 there too.

In this situation I think the only sure-fire way to cope is to use unsigned char[42] in the parts where it matters. Start by writing a precise specification of exactly what fields are where in this 42-byte block, and what endian, then use that definition to write some code to translate between that and a struct you can interact with. The code will likely be either all-at-once serialisation code (aka marshalling), or a bunch of getters and setters.