Spawn process from multithreaded application

Question

I have a situation where I need to spawn a helper process from within a very large, multithreaded application, which I do not have complete control over.

Right now I\'m

Answer 1

fork'ing multithreaded application is considered to be safe if you use only async-signal-safe operations. POSIX says:

A process shall be created with a single thread. If a multi-threaded process calls fork(), the new process shall contain a replica of the calling thread and its entire address space, possibly including the states of mutexes and other resources. Consequently, to avoid errors, the child process may only execute async-signal-safe operations until such time as one of the exec functions is called. Fork handlers may be established by means of the pthread_atfork() function in order to maintain application invariants across fork() calls.

posix_spawn() is not the best idea:

It is also complicated to modify the environment of a multi-threaded process temporarily, since all threads must agree when it is safe for the environment to be changed. However, this cost is only borne by those invocations of posix_spawn() and posix_spawnp() that use the additional functionality. Since extensive modifications are not the usual case, and are particularly unlikely in time-critical code, keeping much of the environment control out of posix_spawn() and posix_spawnp() is appropriate design.

(see man posix_spawn)

I guess you have problems with replicated from parent resources. You may clean them up using pthread_atfork() handler (you use pthread, right?). The other way is to use low level function for process creation called clone(). It gives you almost full control on what exactly child process should inherit from its parent.

[UPDATE]

Probably the simplest way to get rid of the problem is to change your fork'ing scheme. For example you can create a new process (fork) even before your program initializes all resources. I.e. call fork() in main() before you create all your threads. In child process setup a signal handler (for example for SIGUSR2 signal) and sleep. When parent needs to exec some new process, it sends the SIGUSR2 signal to your child process. When child catches it, it calls fork/exec.

Answer 2

Calling fork should be safe if you limit yourself to "raw" system calls (syscall(SYS_fork), syscalll(SYS_execve, ...), etc.). Call into any glibc routine, and you'll be in a lot of trouble.

Calling vfork is not at all what you want: only the thread that called vfork is suspended, and other threads will continue to run (and in the same address space as the vforked child). This is very likely to complicate your life.

Calling clone directly is possible, but exceedingly tricky. We have an implementation that allows for safe forking of child processes from multithreaded apps (unfortunately not open source). That code is very tricky, and surprisingly long.