Maximum number of threads per process in Linux?

Question

What is the maximum number of threads that can be created by a process under Linux?

How (if possible) can this value be modified?

Answer 1

Use nbio non-blocking i/o library or whatever, if you need more threads for doing I/O calls that block

Answer 2

@dragosrsupercool

Linux doesn't use the virtual memory to calculate the maximum of thread, but the physical ram installed on the system

 max_threads = totalram_pages / (8 * 8192 / 4096);

http://kavassalis.com/2011/03/linux-and-the-maximum-number-of-processes-threads/

kernel/fork.c

/* The default maximum number of threads is set to a safe
 * value: the thread structures can take up at most half
 * of memory.
 */
max_threads = mempages / (8 * THREAD_SIZE / PAGE_SIZE);

So thread max is different between every system, because the ram installed can be from different sizes, I know Linux doesn't need to increase the virtual memory, because on 32 bit we got 3 GB for user space and 1 GB for the kernel, on 64 bit we got 128 TB of virtual memory, that happen on Solaris, if you want increase the virtual memory you need to add swap space.

Answer 3

Thread count limit:

$ cat /proc/sys/kernel/threads-max 

How it is calculated:

max_threads = mempages / (8 * THREAD_SIZE / PAGE_SIZE);

and: x86_64 page size (PAGE_SIZE) is 4K; Like all other architectures, x86_64 has a kernel stack for every active thread. These thread stacks are THREAD_SIZE (2*PAGE_SIZE) big;

for mempages :

cat /proc/zoneinfo | grep spanned | awk '{totalpages=totalpages+$2} END {print totalpages}';

so actually the number is not related with limitation of thread memory stack size (ulimit -s).

P.S: thread memory stack limitation is 10M in my rhel VM, and for 1.5G memory, this VM can only afford 150 threads?

Answer 4

To retrieve it:

cat /proc/sys/kernel/threads-max

To set it:

echo 123456789 > /proc/sys/kernel/threads-max

123456789 = # of threads

Answer 5

In practical terms, the limit is usually determined by stack space. If each thread gets a 1MB stack (I can't remember if that is the default on Linux), then you a 32-bit system will run out of address space after 3000 threads (assuming that the last gb is reserved to the kernel).

However, you'll most likely experience terrible performance if you use more than a few dozen threads. Sooner or later, you get too much context-switching overhead, too much overhead in the scheduler, and so on. (Creating a large number of threads does little more than eat a lot of memory. But a lot of threads with actual work to do is going to slow you down as they're fighting for the available CPU time)

What are you doing where this limit is even relevant?

Answer 6

We can see the maximum number of threads defined in the following file in linux

cat /proc/sys/kernel/threads-max

(OR)

sysctl -a | grep threads-max