How to use dlsym reliably when you have duplicated symbols?

Question

Good evening, I'm currently working on a Plugin system in C++/Linux based on the Plux.net model.

To keep it simple, I basicly declare a symbol (lets call it pluginInformation) with extern C (to unmangle) and my plugin manager look for that symbol in the pre configured imports (.so).

The thing is that the main application declares the same symbol, not only that but any dependency it have may have the symbol aswell. (since in this pluginInformation, modules can publish plugs and/or slots).

So when my PluginManager starts, it first try to find the symbol in the main program (passing NULL to dlopen), then it tries to find the symbol in any of its dependencies (using dl_iterate_phdr). And last it will dlopen a set of configure imports (it will read the path of the .so that the user configured, dlopen them and finally dlsym the pluginInformation symbol).

The collection of pluginInformation found in all the modules is used then to build the extension three.

If I declare the symbol in the main program and load the imports using dlopen, it works (as long as I pass the flag RTLD_DEEPBIND when dlopening the imports).

But for the application dependencies I dont have the option of passing the flag (I can but it doesnt do anything) since this .sos were loaded at the start up of the application.

Now when I try to use any of the symbols I got from the dependencies (the ones loaded at start up) I get a segmentation fault. I assume the problem is that I have several symbols with the same name in the symbol table, the weird thing is that it seems to correctly identify that there are several symbols and it even gives me the correct path of the .so where the symbol is declared, but as soon as I access the symbol a segmentation fault occurs. If I only declare the symbol in the main program or in one of the dependencies everything works correctly.

How can I manage duplicate symbols between the main program and the strat up imports with dlsym?.

I have been thinking in keep the mangling and then just try to find my symbol pasring the symbol table, but im not sure this is even possible (listing all the symbols in a module programmatically).

PD: Sorry I didnt post any code, but im not at home right now, I hope the description of what im trying to do is clear enough, if not I can post some code tomorrow.

Answer 1

Here is an alternate approach.

The application itself exports one or more plugin item registration functions. For example:

int register_plugin_item(const char *const text,
                         const char *const icon,
                         void (*enter)(void *),
                         void (*click)(void *),
                         void (*leave)(void *),
                         void *data);

Per registered item, there are two string slots (text and icon), three function slots (enter, click, and leave), and an opaque reference that is given to the functions as a parameter when called.

(Note that you'll need to use the -rdynamic compiler option when compiling the main application (the object file implementing the above function), to make sure the linker adds the register_plugin_item symbol to the dynamic symbol table.)

Each plugin calls the register_plugin_item() function for each of the items it wants, in a constructor function (that is automatically run at library load time). It is possible, and often useful, for the function to first examine the environment it runs in to determine which features to register, or which optimized variants of functions to use for each plugin item.

Here is a trivial example plugin. Note how all the symbols are static, so that the plugin does not pollute the dynamic symbol table, or cause any symbol conflicts.

#include <stdlib.h>
#include <stdio.h>

extern int register_plugin_item(const char *const,
                                const char *const,
                                void (*enter)(void *),
                                void (*click)(void *),
                                void (*leave)(void *),
                                void *);

static void enter(void *msg)
{
    fprintf(stderr, "Plugin: Enter '%s'\n", (char *)msg);
}

static void leave(void *msg)
{
    fprintf(stderr, "Plugin: Leave '%s'\n", (char *)msg);
}

static void click(void *msg)
{
    fprintf(stderr, "Plugin: Click '%s'\n", (char *)msg);
}

static void init(void) __attribute__((constructor));
static void init(void)
{
    register_plugin_item("one", "icon-one.gif",
                         enter, leave, click,
                         "1");

    register_plugin_item("two", "icon-two.gif",
                         enter, leave, click,
                         "2");
}

The above plugin exports two items. For testing, create at least a couple of variants of the above; you will see that there are no symbol conflicts even if the plugins use same (static) variables and function names.

Here is an example application that loads the specified plugins, and tests each registered item:

#include <stdlib.h>
#include <dlfcn.h>
#include <string.h>
#include <errno.h>
#include <stdio.h>

struct item {
    struct item *next;
    const char  *text;
    const char  *icon;
    void        *data;
    void       (*enter)(void *);
    void       (*leave)(void *);
    void       (*click)(void *);
};

static struct item *list = NULL;

int register_plugin_item(const char *const text,
                         const char *const icon,
                         void (*enter)(void *),
                         void (*click)(void *),
                         void (*leave)(void *),
                         void *data)
{
    struct item *curr;

    curr = malloc(sizeof *curr);
    if (!curr)
        return ENOMEM;

    curr->text = text;
    curr->icon = icon;
    curr->data = data;
    curr->enter = enter;
    curr->leave = leave;
    curr->click = click;

    /* Prepend to list */
    curr->next = list;
    list = curr;

    return 0;
}

int main(int argc, char *argv[])
{
    int          arg;
    void        *handle;
    struct item *curr;

    if (argc < 2 || !strcmp(argv[1], "-h") || !strcmp(argv[1], "--help")) {
        fprintf(stderr, "\n");
        fprintf(stderr, "Usage: %s [ -h | --help ]\n", argv[0]);
        fprintf(stderr, "       %s PLUGIN.so ... \n", argv[0]);
        fprintf(stderr, "\n");
        fprintf(stderr, "Please supply full plugin paths, unless\n");
        fprintf(stderr, "the plugins reside in a standard library directory,\n");
        fprintf(stderr, "or in a directory listed in LD_LIBRARY_PATH.\n");
        fprintf(stderr, "\n");
        return 1;
    }

    for (arg = 1; arg < argc; arg++) {

        handle = dlopen(argv[arg], RTLD_NOW);
        if (handle != NULL)
            fprintf(stderr, "%s: Loaded.\n", argv[arg]);
        else
            fprintf(stderr, "%s.\n", dlerror());

        /* Note: We deliberately "leak" the handle,
         *       so that the plugin is not unloaded. */
    }

    for (curr = list; curr != NULL; curr = curr->next) {
        if (curr->text)
            printf("Item '%s':\n", curr->text);
        else
            printf("Unnamed item:\n");

        if (curr->icon)
            printf("\tIcon is '%s'\n", curr->icon);
        else
            printf("\tNo icon\n");

        if (curr->data)
            printf("\tCustom data at %p\n", curr->data);
        else
            printf("\tNo custom data\n");

        if (curr->enter)
            printf("\tEnter handler at %p\n", curr->enter);
        else
            printf("\tNo enter handler\n");

        if (curr->click)
            printf("\tClick handler at %p\n", curr->click);
        else
            printf("\tNo click handler\n");

        if (curr->leave)
            printf("\tLeave handler at %p\n", curr->leave);
        else
            printf("\tNo leave handler\n");

        if (curr->enter || curr->click || curr->leave) {
            printf("\tTest calls:\n");
            if (curr->enter)
                curr->enter(curr->data);
            if (curr->click)
                curr->click(curr->data);
            if (curr->leave)
                curr->leave(curr->data);
            printf("\tTest calls done.\n");
        }
    }

    return 0;
}

If the application is app.c, and you have plugins plugin-foo.c and plugin-bar.c, you can compile them using e.g.

gcc -W -Wall -rdynamic app.c -ldl -o app

gcc -W -Wall -fpic -c plugin-foo.c
gcc -shared -Wl,-soname,plugin-foo.so plugin-foo.o -o plugin-foo.so

gcc -W -Wall -fpic -c plugin-bar.c
gcc -shared -Wl,-soname,plugin-bar.so plugin-bar.o -o plugin-bar.so

and run using e.g.

./app --help

./app ./plugin-foo.so

./app ./plugin-foo.so ./plugin-bar.so

Note that if the same plugin is defined more than once, the constructor is only executed once for that library. There will be no duplicate registrations.

---

The interface between the plugins and the application is completely up to you. In this example, there is only one function. A real application would probably have more. The application can also export other functions, for example for the plugin to query application configuration.

Designing a good interface is a whole different topic, and definitely deserves at least as much thought as you put in the implementation.

The Plux.NET plugin platform allows plugins to export their own slots, too. This alternate approach allows that in many ways. One of them is to export a plugin registration function -- that is, for registering plugins instead of individual items -- that takes a function pointer:

int register_plugin(const char *const name,
                    int (*extend)(const char *const, ...));

If the plugin provides slots, it provides its own registration function as the extend function pointer. The application also exports a function, for example

int plugin_extend(const char *const name, ...);

that the plugins can use to call other plugins' registration functions. (The implementation of plugin_extend() in the main application involves searching for a suitable extend function already registered, then calling it/them.)

Implementation-wise, allowing plugins to export slots complicates implementation quite a bit. In particular, when and in which order should the slots exported by the plugins become available? Is there a specific order in which plugins must be loaded, to make sure all possible slots are exported? What happens if there is a circular dependency? Should plugins specify which other plugins they rely on before the registrations commence?

If each plugin is a separate entity that does not export any slots of its own, only plugs into main application slots, you avoid most of the complexity in the implementation.

The order in which registered items are examined is a detail you probably need to think about, though. The above example program uses a linked list, in which the items end up in reverse order compared to the registration order, and registration order is the same as the order in which the plugin file names are first specified on the command line. If you have a plugin directory, which is automatically scanned (using e.g. opendir()/readdir()/dlopen()/closedir() loop), then the plugin registration order is semi-random (depending on the filesystem; usually changing only when plugins are added or removed).

Corrections? Questions? Comments?