Static allocation of opaque data types

Question

Very often malloc() is absolutely not allowed when programming for embedded systems. Most of the time I\'m pretty able to deal with this, but one thing irritates me: it keep

Answer 1

I'm a little confused why you say you can't use malloc(). Obviously on an embedded system you have limited memory and the usual solution is to have your own memory manager which mallocs a large memory pool and then allocates chunks of this out as needed. I've seen various different implementations of this idea in my time.

To answer your question though, why don't you simply statically allocate a fixed size array of them in module.c add an "in-use" flag, and then have create_handle() simply return the pointer to the first free element.

As an extension to this idea, the "handle" could then be an integer index rather than the actual pointer which avoids any chance of the user trying to abuse it by casting it to their own definition of the object.

Answer 2

This is an old question, but since it's also biting me, I wanted to provide here a possible answer (which I'm using).

So here is an example :

// file.h
typedef struct { size_t space[3]; } publicType;
int doSomething(publicType* object);

// file.c
typedef struct { unsigned var1; int var2; size_t var3; } privateType;

int doSomething(publicType* object)
{
    privateType* obPtr  = (privateType*) object;
    (...)
}

Advantages : publicType can be allocated on stack.

Note that correct underlying type must be selected in order to ensure proper alignment (i.e. don't use char). Note also that sizeof(publicType) >= sizeof(privateType). I suggest a static assert to make sure this condition is always checked. As a final note, if you believe your structure may evolve later on, don't hesitate to make the public type a bit bigger, to keep room for future expansions without breaking ABI.

Disadvantage : The casting from public to private type can trigger strict aliasing warnings.

I discovered later on that this method has similarities with struct sockaddr within BSD socket, which meets basically the same problem with strict aliasing warnings.

Answer 3

The least grim solution I've seen to this has been to provide an opaque struct for the caller's use, which is large enough, plus maybe a bit, along with a mention of the types used in the real struct, to ensure that the opaque struct will be aligned well enough compared to the real one:

struct Thing {
    union {
        char data[16];
        uint32_t b;
        uint8_t a;
    } opaque;
};
typedef struct Thing Thing;

Then functions take a pointer to one of those:

void InitThing(Thing *thing);
void DoThingy(Thing *thing,float whatever);

Internally, not exposed as part of the API, there is a struct that has the true internals:

struct RealThing {
    uint32_t private1,private2,private3;
    uint8_t private4;
};
typedef struct RealThing RealThing;

(This one just has uint32_t' anduint8_t' -- that's the reason for the appearance of these two types in the union above.)

Plus probably a compile-time assert to make sure that RealThing's size doesn't exceed that of Thing:

typedef char CheckRealThingSize[sizeof(RealThing)<=sizeof(Thing)?1:-1];

Then each function in the library does a cast on its argument when it's going to use it:

void InitThing(Thing *thing) {
    RealThing *t=(RealThing *)thing;

    /* stuff with *t */
}

With this in place, the caller can create objects of the right size on the stack, and call functions against them, the struct is still opaque, and there's some checking that the opaque version is large enough.

One potential issue is that fields could be inserted into the real struct that mean it requires an alignment that the opaque struct doesn't, and this won't necessarily trip the size check. Many such changes will change the struct's size, so they'll get caught, but not all. I'm not sure of any solution to this.

Alternatively, if you have a special public-facing header(s) that the library never includes itself, then you can probably (subject to testing against the compilers you support...) just write your public prototypes with one type and your internal ones with the other. It would still be a good idea to structure the headers so that the library sees the public-facing Thing struct somehow, though, so that its size can be checked.

Answer 4

I faced a similar problem in implementing a data structure in which the header of the data structure, which is opaque, holds all the various data that needs to be carried over from operation to operation.

Since re-initialization might cause a memory leak, I wanted to make sure that data structure implementation itself never actually overwrite a point to heap allocated memory.

What I did is the following:

/** 
 * In order to allow the client to place the data structure header on the
 * stack we need data structure header size. [1/4]
**/
#define CT_HEADER_SIZE  ( (sizeof(void*) * 2)           \
                        + (sizeof(int) * 2)             \
                        + (sizeof(unsigned long) * 1)   \
                        )

/**
 * After the size has been produced, a type which is a size *alias* of the
 * header can be created. [2/4] 
**/        
struct header { char h_sz[CT_HEADER_SIZE]; };
typedef struct header data_structure_header;

/* In all the public interfaces the size alias is used. [3/4] */
bool ds_init_new(data_structure_header *ds /* , ...*/);

In the implementation file:

struct imp_header {
    void *ptr1, 
         *ptr2;
    int  i, 
         max;
    unsigned long total;
};

/* implementation proper */
static bool imp_init_new(struct imp_header *head /* , ...*/)
{
    return false; 
}

/* public interface */
bool ds_init_new(data_structure_header *ds /* , ...*/) 
{
    int i;

    /* only accept a zero init'ed header */
    for(i = 0; i < CT_HEADER_SIZE; ++i) {
        if(ds->h_sz[i] != 0) {
            return false;
        }
    }

    /* just in case we forgot something */
    assert(sizeof(data_structure_header) == sizeof(struct imp_header));

    /* Explicit conversion is used from the public interface to the
     * implementation proper.  [4/4]
     */
    return imp_init_new( (struct imp_header *)ds /* , ...*/); 
}

client side:

int foo() 
{
    data_structure_header ds = { 0 };

    ds_init_new(&ds /*, ...*/);
}

Answer 5

It is simple, simply put the structs in a privateTypes.h header file. It will not be opaque anymore, still, it will be private to the programmer, since it is inside a private file.

An example here: Hiding members in a C struct

Answer 6

One solution if to create a static pool of struct handle_t objects, and provide then as neceessary. There are many ways to achieve that, but a simple illustrative example follows:

// In file module.c
struct handle_t 
{
    int foo;
    void* something;
    int another_implementation_detail;

    int in_use ;
} ;

static struct handle_t handle_pool[MAX_HANDLES] ;

handle_t* create_handle() 
{
    int h ;
    handle_t* handle = 0 ;
    for( h = 0; handle == 0 && h < MAX_HANDLES; h++ )
    {
        if( handle_pool[h].in_use == 0 )
        {
            handle = &handle_pool[h] ;
        }
    }

    // other initialization
    return handle;
}

void release_handle( handle_t* handle ) 
{
    handle->in_use = 0 ;
}

There are faster faster ways of finding an unused handle, you could for example keep a static index that increments each time a handle is allocated and 'wraps-around' when it reaches MAX_HANDLES; this would be faster for the typical situation where several handles are allocated before releasing any one. For a small number of handles however, this brute-force search is probably adequate.

Of course the handle itself need no longer be a pointer but could be a simple index into the hidden pool. This would enhance data hiding and protection of the pool from external access.

So the header would have:

typedef int handle_t ;

and the code would change as follows:

// In file module.c
struct handle_s 
{
    int foo;
    void* something;
    int another_implementation_detail;

    int in_use ;
} ;

static struct handle_s handle_pool[MAX_HANDLES] ;

handle_t create_handle() 
{
    int h ;
    handle_t handle = -1 ;
    for( h = 0; handle != -1 && h < MAX_HANDLES; h++ )
    {
        if( handle_pool[h].in_use == 0 )
        {
            handle = h ;
        }
    }

    // other initialization
    return handle;
}

void release_handle( handle_t handle ) 
{
    handle_pool[handle].in_use = 0 ;
}

Because the handle returned is no longer a pointer to the internal data, and inquisitive or malicious user cannnot gain access to it through the handle.

Note that you may need to add some thread-safety mechanisms if you are getting handles in multiple threads.