libusb-1.0 hotplug events stop working in parent after fork(), when child calls libusb_exit()

Question

I've been developing an application that monitors the USB device tree using libusb_hotplug_register_callback(). When a device that matches some criteria is attached, it will fork() and exec() an application to handle this device.

The application has been working fine for some time now, but I've come back to try and 'tidy it up'...

libusb will open a number of file descriptors (see below) which it monitors for events, etc... The problem is that after I call fork() and before I call exec(), I'd like to shutdown libusb, closing the file descriptors and leaving the children in a clean state.

Parent:

root@imx6q:~# ls -l /proc/14245/fd
total 0
lrwx------ 1 root root 64 Feb  9 18:15 0 -> /dev/pts/2
lrwx------ 1 root root 64 Feb  9 18:15 1 -> /dev/pts/2
lrwx------ 1 root root 64 Feb  9 18:15 2 -> /dev/pts/2
lrwx------ 1 root root 64 Feb  9 18:15 3 -> socket:[1681018]
lr-x------ 1 root root 64 Feb  9 18:15 4 -> pipe:[1681019]
l-wx------ 1 root root 64 Feb  9 18:15 5 -> pipe:[1681019]
lr-x------ 1 root root 64 Feb  9 18:15 6 -> pipe:[1681020]
l-wx------ 1 root root 64 Feb  9 18:15 7 -> pipe:[1681020]
lrwx------ 1 root root 64 Feb  9 18:15 8 -> anon_inode:[timerfd]

Child:

root@imx6q:~# ls -l /proc/14248/fd
total 0
lr-x------ 1 root root 64 Feb  9 18:15 0 -> /dev/null
l-wx------ 1 root root 64 Feb  9 18:15 1 -> /dev/null
lrwx------ 1 root root 64 Feb  9 18:15 2 -> /dev/pts/2
lr-x------ 1 root root 64 Feb  9 18:15 4 -> pipe:[1681019]
l-wx------ 1 root root 64 Feb  9 18:15 5 -> pipe:[1681019]
lr-x------ 1 root root 64 Feb  9 18:15 6 -> pipe:[1681020]
l-wx------ 1 root root 64 Feb  9 18:15 7 -> pipe:[1681020]
lrwx------ 1 root root 64 Feb  9 18:15 8 -> anon_inode:[timerfd]

The issue I've come up against, is that by calling libusb_exit() in the child, the parent no longer sees any hotplug events.

I've tried re-registering my callback after a fork() (in the parent) with no luck (and no errors).

I've had a bit of a rummage in the libusb and libudev sources, and I'm wondering if this is a side-effect of libusb's linux_udev_stop_event_monitor() calling udev_monitor_unref()... But alas there is no 'communication' there, just a call to close() when the ref count reaches zero. And anyway, the missing socket from above is most likely the netlink socket that udev opens (politely, with SOCK_CLOEXEC).

Below is an example application that demonstrates the issue:

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <assert.h>
#include <libusb-1.0/libusb.h>

libusb_context *libusb_ctx;
libusb_hotplug_callback_handle cb_handle;

int hotplug_callback(struct libusb_context *ctx, struct libusb_device *dev, libusb_hotplug_event event, void *user_data) {
    pid_t pid;
    char *cmd[] = {
        "sleep", "600", NULL
    };

    fprintf(stderr, "event! %d\n", event);

    /* fork, and return if in parent */
    pid = fork();
    assert(pid != -1);
    if (pid != 0) {
        fprintf(stderr, "intermediate child's PID is: %d\n", pid);
        return 0;
    }

    /* setsid() and re-fork() to complete daemonization */
    assert(setsid() != -1);
    pid = fork();
    assert(pid != -1);
    if (pid != 0) {
        fprintf(stderr, "child's PID is: %d\n", pid);
        exit(0);
    }

#if 1 /* toggle this */
    fprintf(stderr, "libusb is NOT shutdown in child...\n");
#else
    /* shutdown libusb */
    libusb_hotplug_deregister_callback(libusb_ctx, cb_handle);
    libusb_exit(libusb_ctx);
    fprintf(stderr, "libusb is shutdown in child...\n");
#endif

    /* now that the child has reached this point, you'll never see a hotplug event again! */

    /* exec() */
    assert(execvp(cmd[0], cmd) == 0);
    abort();
}

void main(void) {
    pid_t pid;

    /* setup libusb & hotplug callback */
    assert(libusb_init(&libusb_ctx) == 0);
    assert(libusb_hotplug_register_callback(libusb_ctx, LIBUSB_HOTPLUG_EVENT_DEVICE_ARRIVED, LIBUSB_HOTPLUG_NO_FLAGS, LIBUSB_HOTPLUG_MATCH_ANY,
                                            LIBUSB_HOTPLUG_MATCH_ANY, LIBUSB_HOTPLUG_MATCH_ANY, hotplug_callback, NULL, &cb_handle) == LIBUSB_SUCCESS);

    pid = getpid();
    fprintf(stderr, "running... parent's PID is: %d\n", pid);

    /* allow libusb to handle events */
    while (1) {
        assert(libusb_handle_events_completed(NULL, NULL) == 0);
    }
}

With the 'toggle this' #if set to 1, I see the following session (3x connections):

# ./main
running... parent's PID is: 14370
event! 1
intermediate child's PID is: 14372
child's PID is: 14373
libusb is NOT shutdown in child...
event! 1
intermediate child's PID is: 14375
child's PID is: 14376
libusb is NOT shutdown in child...
event! 1
intermediate child's PID is: 14379
child's PID is: 14380
libusb is NOT shutdown in child...
^C

With the 'toggle this' #if set to 0, I see the following session (3x connections, only the first is actioned):

# ./main
running... parent's PID is: 14388
event! 1
intermediate child's PID is: 14390
child's PID is: 14391
libusb is shutdown in child...
^C

Software versions are:

• libusb: libusb-1.0.so.0.1.0 / 1.0.20-r1
• libudev: libudev.so.0.13.1 / 182-r7
• kernel: 3.14.38

If anyone has seen this before, or can reproduce it (or can't reproduce it!) I'd appreciate your input. Thanks in advance!

Answer 1

In general I do not recommend calling fork from an portable libusb program unless fork is immediately followed by exec. This is due to known issues with fork on OS X. The issue was discussed on the libusb-devel mailing list some time ago (2002) and it looks like I forgot to add this caveat to the documentation for libusb-1.0. I recommend using threads instead.

On Linux we inherit any fork caveats from libudev. I do not see anything in their documentation about fork so I do not know if it should work or not. You can always try with netlink by adding --disable-udev. This option is not recommended for production use but it might help isolate the problem.