unshare --pid /bin/bash - fork cannot allocate memory

Question

I'm experimenting with linux namespaces. Specifically the pid namespace.

I thought I'd test something out with bash but run into this problem:

unshare -p /bin/bash
bash: fork: Cannot allocate memory

Running ls from there gave a core dump. Exit is the only thing possible.

Why is it doing that?

Answer 1

The error is caused by the PID 1 process exits in the new namespace.

After bash start to run, bash will fork several new sub-processes to do somethings. If you run unshare without -f, bash will have the same pid as the current "unshare" process. The current "unshare" process call the unshare systemcall, create a new pid namespace, but the current "unshare" process is not in the new pid namespace. It is the desired behavior of linux kernel: process A creates a new namespace, the process A itself won't be put into the new namespace, only the sub-processes of process A will be put into the new namespace. So when you run:

unshare -p /bin/bash

The unshare process will exec /bin/bash, and /bin/bash forks several sub-processes, the first sub-process of bash will become PID 1 of the new namespace, and the subprocess will exit after it completes its job. So the PID 1 of the new namespace exits.

The PID 1 process has a special function: it should become all the orphan processes' parent process. If PID 1 process in the root namespace exits, kernel will panic. If PID 1 process in a sub namespace exits, linux kernel will call the disable_pid_allocation function, which will clean the PIDNS_HASH_ADDING flag in that namespace. When linux kernel create a new process, kernel will call alloc_pid function to allocate a PID in a namespace, and if the PIDNS_HASH_ADDING flag is not set, alloc_pid function will return a -ENOMEM error. That's why you got the "Cannot allocate memory" error.

You can resolve this issue by use the '-f' option:

unshare -fp /bin/bash

If you run unshare with '-f' option, unshare will fork a new process after it create the new pid namespace. And run /bin/bash in the new process. The new process will be the pid 1 of the new pid namespace. Then bash will also fork several sub-processes to do some jobs. As bash itself is the pid 1 of the new pid namespace, its sub-processes can exit without any problem.

Answer 2

This does not explain why this happens, but shows how to correctly launch a shell in a new pid namespace:

Use the -f flag to fork off the shell from unshare:

unshare -fp /bin/bash

You also need to pass the --mount-proc option to ensure that the shell will get PID 1 in the newly created namespace:

unshare -fp --mount-proc /bin/bash

Now run ps:

# ps
   PID TTY          TIME CMD
 1 pts/1    00:00:00 bash
11 pts/1    00:00:00 ps